{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Non-linear Regression Demo\n",
    "\n",
    "_Source: 🤖[Homemade Machine Learning](https://github.com/trekhleb/homemade-machine-learning) repository_\n",
    "\n",
    "> ☝Before moving on with this demo you might want to take a look at:\n",
    "> - 📗[Math behind the Linear Regression](https://github.com/trekhleb/homemade-machine-learning/tree/master/homemade/linear_regression)\n",
    "> - ⚙️[Linear Regression Source Code](https://github.com/trekhleb/homemade-machine-learning/blob/master/homemade/linear_regression/linear_regression.py)\n",
    "\n",
    "**Polynomial regression** is a form of regression analysis in which the relationship between the independent variable `x` and the dependent variable `y` is modelled as an _n<sup>th<sup>_ degree polynomial in `x`. Although polynomial regression fits a nonlinear model to the data, as a statistical estimation problem it is linear, in the sense that the regression function `E(y|x)` is linear in the unknown parameters that are estimated from the data. For this reason, polynomial regression is considered to be a special case of multiple linear regression.\n",
    "\n",
    "> **Demo Project:** In this example we will train our model to imitate an artificial non-linear equation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To make debugging of linear_regression module easier we enable imported modules autoreloading feature.\n",
    "# By doing this you may change the code of linear_regression library and all these changes will be available here.\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "# Add project root folder to module loading paths.\n",
    "import sys\n",
    "sys.path.append('../..')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Dependencies\n",
    "\n",
    "- [pandas](https://pandas.pydata.org/) - library that we will use for loading and displaying the data in a table\n",
    "- [numpy](http://www.numpy.org/) - library that we will use for linear algebra operations\n",
    "- [matplotlib](https://matplotlib.org/) - library that we will use for plotting the data\n",
    "- [linear_regression](https://github.com/trekhleb/homemade-machine-learning/blob/master/src/linear_regression/linear_regression.py) - custom implementation of linear regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import 3rd party dependencies.\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Import custom linear regression implementation.\n",
    "from homemade.linear_regression import LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the Data\n",
    "\n",
    "In this demo we will use artificial non-linear data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>y</th>\n",
       "      <th>x</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>97.58776</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>97.76344</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>96.56705</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>92.52037</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>91.15097</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>95.21728</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>90.21355</td>\n",
       "      <td>7.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>89.29235</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>91.51479</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>89.60966</td>\n",
       "      <td>10.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          y     x\n",
       "0  97.58776   1.0\n",
       "1  97.76344   2.0\n",
       "2  96.56705   3.0\n",
       "3  92.52037   4.0\n",
       "4  91.15097   5.0\n",
       "5  95.21728   6.0\n",
       "6  90.21355   7.0\n",
       "7  89.29235   8.0\n",
       "8  91.51479   9.0\n",
       "9  89.60966  10.0"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load the data.\n",
    "data = pd.read_csv('../../data/non-linear-regression-x-y.csv')\n",
    "\n",
    "# Fetch traingin set and labels.\n",
    "x = data['x'].values.reshape((data.shape[0], 1))\n",
    "y = data['y'].values.reshape((data.shape[0], 1))\n",
    "\n",
    "# Print the data table.\n",
    "data.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot the Data\n",
    "\n",
    "Let's visualize the training and test datasets to see the shape of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl8XHW5+PHPM5N93/ekSZo03TfSDUqhbAVEFkUuoAIXFK8i6nXF6/WnXl96RS8qKqIISN0KCCiL7JXKVtqmdF+TNvu+7/t8f3+cSUj3NJnJTGae9+vVV2fOnDnnOZ30yXee813EGINSSinfZfN0AEoppdxLE71SSvk4TfRKKeXjNNErpZSP00SvlFI+ThO9Ukr5uDMmehF5VEQaRGTvSV77iogYEUlwPhcR+YWIlIjIbhFZ6o6glVJKjd94WvSPAZcfv1FEMoHLgIoxm68A8p1/7gQenHyISimlJuOMid4Y8ybQcpKXfgZ8HRg74uoa4A/G8h4QIyKpLolUKaXUhARM5E0icg1QbYzZJSJjX0oHKsc8r3Juqz3d8RISEkx2dvZEQlFKKb+1ffv2JmNM4pn2O+tELyJhwH9hlW0mTETuxCrvkJWVRVFR0WQOp5RSfkdEysez30R63cwEcoBdIlIGZADvi0gKUA1kjtk3w7ntBMaYh4wxhcaYwsTEM/5CUkopNUFnneiNMXuMMUnGmGxjTDZWeWapMaYOeA64xdn7ZiXQbow5bdlGKaWUe42ne+UGYDNQICJVInLHaXZ/ETgKlAC/Az7nkiiVUkpN2Blr9MaYm87wevaYxwa4a/JhKaWUchUdGauUUj5OE71SSvk4TfRKKeXjJjRgSilvVNLQycYDDWQnhHPR7CQC7dqOUQo00Ssf8cjbpXz/hf2jz//3Iwu4aXmWByNSyntok0dNe2VN3dz78kHWFiTyzj0XER5k51Bdp6fDUspraIteTTu17b38/p0yqtt6qWvvo6Shi2C7jR99dCHJUSHkJIZztKnb02Eq5TU00atp55f/LOHxrRXMiA8nOSqYy+elcN3SdJKjQgDITYjg/YpWD0eplPfQRK+mjR+/fJD4iGCe21nDdUsyuO+GRSfdLzcxnOd319A3OExIoH2Ko1TK+2iiV9OCMYb175bRPTAMwL8tyzzlvrmJERgDZc3dzE6JmqoQlfJaejNWTQsdvUN0DwwTERzAwoxolmXHnnLf3IRwAI42ap1eKdAWvZomqtt6Abj3owu5dG4yxy14c4zcxJFE3zUlsSnl7bRFr6aFkUSfHhtKUMDpf2zDggLIiA1lW5nekFUKNNErL9bRN0hxvdUfvsaZ6NNiQsb13o+dk8m/Djdqf3ql0ESvvNgvNxZz5S/e4nB9JzVtvQQF2EgIDx7Xe29ZNYPQQDu//dcRN0eplPfTRK+81r6aDgaHDfc8vZvK1h7SokOw2U5dmx8rNjyIG5dn8tyumtGyj1L+ShO98lqH6ztJigzm/Yo2Xj/QQHps6Fm9/1Pn5wLw8FtH3RGeUtOGJnrllZq6+mnqGuD21TmkRIUwMOQgLfrsEn16TChXL0rj8a2V9Dr73yvljzTRK6902HkTdm5qFB9fYc1CmRZzdoke4MLZSfQODlPeon3qlf/SRK+8UnG91Qe+ICWSG5dnkR4TyjkzTj1I6lQyneWeyhat0yv/pQOmlFc6VN9JdGggSZHBiAjv3HPRhI6TGRcGQGVLjyvDU2paOWOLXkQeFZEGEdk7ZttPROSgiOwWkb+JSMyY174pIiUickhE1rkrcOXb9la3U5AcedoRsOMRHx5EaKCdipYePv7we7y4p9ZFESo1fYyndPMYcPlx214D5htjFgKHgW8CiMhc4EZgnvM9vxYRnT5QnZW69j52V7VzQUHipI8lImTGhfLm4UbeKWnmzcONLohQqenljIneGPMm0HLctleNMUPOp+8BGc7H1wCPG2P6jTGlQAmw3IXxKj/w6v46ANbNS3bJ8TJjw0YXItE+9cofueJm7O3AS87H6UDlmNeqnNuUGrdX9tUxMzGcvKRIlxxvpE4PmuiVf5pUoheRbwFDwJ8n8N47RaRIRIoaG/XrtLJsPtLMu0eauXJBqsuOOTbR17T14nAYHA7jsuMr5e0mnOhF5DbgKuDjxpiR/zXVwNgVITKc205gjHnIGFNojClMTJx8LVZNf+29g3zpiR3kxIfzHxfMdNlxR7pYJkUG0zfo4N5XDnLRfZtcdnylvN2EEr2IXA58HbjaGDO239pzwI0iEiwiOUA+sHXyYSp/8M+D9dR39POjjy4kPNh1PX8XZcaQFRfGJ1fOAGDDlgrKmnvo6h86wzuV8g3j6V65AdgMFIhIlYjcAfwKiAReE5GdIvIbAGPMPuBJYD/wMnCXMUbHnqtxeau4idiwQAonMDDqdJKjQnjz62tZOzsJgI4+K8E3dPS59DxKeaszNpuMMTedZPMjp9n/B8APJhOU8j/GGN4qbmJ1fuK4Z6g8WxnHTYpW39FPbmKEW86llDfRKRCUVzhU30ljZz/n5ye47RzRoYGEB30wrKNeW/TKT2iiV17h2Z01AG5N9CJCWkwoCRFBgCZ65T800SuPq2rt4ZG3S7l2cRqpZzkV8dm6Y3UOX798NuFBduo7+t16LqW8hU5qpjzuN/86gk3g65fPdvu5blxuTXn8m01HqO/UFr3yD9qiVx53oLaTxZkxE5pvfqKSooK1143yG5rolcdVtPSQNWb06lRIjgrR0o3yG5rolUf1DgzT2NnvoUTfxweDupXyXZrolUdVtVoDqzOnONEnRQbTP+Sgo1dHxyrfp4leeVSlhxL9yOCpn71+mIEhx5SeW6mppoleeVRFs5Xop7p0c/GcZG5ekcVj75bx9PtVU3pupaaaJnrlURUtvYQG2okPD5rS8wbabfzg2vlEhgSwv6ZjSs+t1FTTRK88qrLV6nEz2bVhJ0JEyE+KoLihc8rPrdRU0kSvPKqiuWfK6/Nj5SdFUtLQ5bHzKzUVNNErjznS2MWh+k6WZMV4LIb85AiaugZo6R7wWAxKuZsmeuUxf9lSQYBNuKEw88w7u8nMJGuaYm3VK1+miV55RP/QME9tr2Ld/BQSI4M9Fke+M9FrnV75Mk30yiNq2/po7x1kbUGSR+NIiw4lLMjOoTpN9Mp3aaJXHtHeOwhATGigR+Ow2YTz8hJ4dmcN3bqGrPJRmuiVR4wk+ugwzyZ6gM9eOJP23kE2bK3wdChKuYUmeuURo4newy16gKVZsSzPjuOJbZWeDkUpt9BErzzCmxI9wNy0KGrbdX565ZvOmOhF5FERaRCRvWO2xYnIayJS7Pw71rldROQXIlIiIrtFZKk7g1fTl7cl+sTIYLr6h+gdGPZ0KEq53Hha9I8Blx+37R5gozEmH9jofA5wBZDv/HMn8KBrwlS+pqN3kKAAGyGBdk+HAkBihNXFs6lLFyNRvueMid4Y8ybQctzma4D1zsfrgWvHbP+DsbwHxIhIqquCVb6jvXfQa1rzwGhf/kZN9MoHTbRGn2yMqXU+rgOSnY/TgbF3tKqc25Q6hrcl+gRni76xUxO98j2TvhlrrLXYzno9NhG5U0SKRKSosbFxsmGoacbbEv1Ii15LN8oXTTTR14+UZJx/Nzi3VwNjJy7JcG47gTHmIWNMoTGmMDExcYJhqOnK2xJ9fIQ1H7626JUvmmiifw641fn4VuDZMdtvcfa+WQm0jynxuNyhuk5++OIBbn10K7sq29x1GuUG3pboA+02YsMCtUWvfNJ4ulduADYDBSJSJSJ3AD8CLhWRYuAS53OAF4GjQAnwO+Bzbonaqaq1h8feKaOorIUvPL5Dh7BPI96W6MEq33hri94YM7rsolJnazy9bm4yxqQaYwKNMRnGmEeMMc3GmIuNMfnGmEuMMS3OfY0x5i5jzExjzAJjTJE7gz8/P5E937uMR25bRkVLDw9uOuLO0ykXGXYYOvuGiPKyRJ8Q4b2JftPhRtb85A1Km7o9HYqahgI8HcBkBAVYv6dW5sazbEYc7x5pAgo8G5Q6rSe3VbJ+cxngPYOlRiRGBrOjwrtKgH/YXEag3UZ9hzVqt6Klh5yEcM8GpaYdn5kCYUFGNPtqOhgadlDd1suFP3mDI426mIQ3McbwmzePsM+5GLe3JfqxLXqrM5lntfcO8sMXD/DgpiMU11s/y01e+o1DeTefSfQLM6LpH3JQ3NDFO8VNlDX3sPlIs6fDUmPsrmrnaOMHpQdvS/RJkcH0Dg6z7mdvMv87r/Ckhyc5+/uOavoGHVS09FBUbo1Z1JvFaiJ8JtEvSI8GYE9VO7urra/fujycd3nm/SqCAmyIWM8jgr2rcnjtknRuOzeb9NhQRIT3Sj3bUNiwtWL036i+w0rwmujVRPhMos+ODycyOIDd1W3sqbZKA7o8nHd5/UADF85K5N6PLAQgIzbUwxEdKzkqhO9ePY9Hb1vGgvRoyqewl8snHt7C/a8Xs6OilU+tL6KsqZuDdZ18Zk3u6C9GgKYuXcRcnT3valJNgs0mzE+P5r2jLVS0WP9BR+qayvOqWnuobuvlU+fncMOyTD68KI3QIO+Y0OxkshPCeG1/PQDF9Z186Ymd/PGOFcSFB7n8XOXN3bxd0kRb7wCtPQO8fqCeyBDrv+bFc5J5fncNh+u7CAuya4teTYjPtOgBrj8ng5KGLgaGHCxIj6ahs5/2nkFq2np587BOs+BJ28qsGvPynDgAr07yADPiw2nqGqCzb5C3ipvYV9PBnup2t5zr9QPWwPJDdZ1sL28F4G87qokKCaAgJZJFGTHYBM6ZEeu13T+Vd/OpRH/dknTOmRELWPVWgJLGTr73/D5u+/1Wqtt6PRmeX9ta2kJUSACzU6I8Hcq4ZMeHAVDe3EOJs/dWZYt7SjkbD9QjAoPD5phfJstz4rDbhLvW5nH/jUvIiA3V0o2aEJ9K9Dab8LMbFvPfH5rDJXOSANhS2sI/DzbgMPDHzeUejtB/bSltYVm2lbimgxnxVl/1suZujjS4L9F39g2ytbSFK+d/MJv3ylzrW8/It5/shHA+vCiNhIhgWrr7+fKTO3nsnVKXx6J8l08leoCs+DA+dX4umbFhpEWH8NNXDzM4bJidEsnj2yp0BSEP6Owb5GhjN0ud37amgxljWvRHnF1CK9yQ6LeVtTDkMNy8Imu0u+k9V8zhsrnJfGhh2jH7JkQE4zDwzPvV/OZfR3E4PN/XX00PPpfoR9hswu9uLSQk0M7c1Cj+68o5tPUMsvlok6dD8zsj3VxnJUd6OJLxCwsKICkymF2VbaM3QN2R6LeUthBoF5ZmxbIgPZqokAAWZUTz0C2FpMcc2ytpZM58gLqOPoqc9XylzsRnet2czLy0aF764vkE2IXYsCCCA2y8VdzERbOTz/xm5TIjiT4vKcLDkZydgpRINjlv4qfHhLqldLPlaAuLMmIIDbLz1XUF1Hf0IXLy8laCcyrlhIgguvqHeG5X9Wh5R6nT8dkW/YjMuDBSo0MJCbSzPCeOt4tP3aL3hmHvvqikoYugABuZXtZv/kw+e8FMBoYcAKydnUhH3xDtPYMuO353/xB7qttZ4azJL86MYd28lFPun+BcHOWi2UlcPCeZF/fUMTTscFk8ynf5fKIf6/z8BIobuqhr7+OBN0r48hM7j0nun/vz+3z1r7s8GKFvKm7oIjchnAD79PpxOzcvgYtmJxEaaOfcmQmAa8s3OyraGHYYlufEj2v/jNhQzs9P4MblWXx4YRot3QO8q9N8qHHw6dLN8VbnJQIHuX/jYf5aVMWQw3Dp3GSuWJCKMYbNR5uPqYMq1yhu6GRRRoynw5iQn9+4mIrmntHeQhUtPSzIiJ7w8Vq6B6hr72NGfBhlzdZN3lnJ4ytpBQfY+eMdKwDoGxwmMjiA53fVsGaWrtCmTm96NbEmaU5qJFctTGXD1krsNiE3IZz/fekgww5Dc/cAbT2DVLX2aAnHRQ7WdfClx3dQ1do77erzI6JCApmfHk12fDh2m3CormPCx3I4DBfft4krf/EWX9iwg9r2Xuw2ISky5KyPFRJo57J5Kby8t85t/fuV7/CrRC8i/Pj6hVwwK5GvXlbAnWtyqWjpobKlZ/SGYd+gQweluMjzu2r4+84ajIGCadTj5mRCg+zMSo5kxySWrKzt6KPVWeM/WNdJbVsfKVEhEx5bcMfqHETg2gfeoUYHA6rT8KtED1a3ufW3L+fTa3KZ6WxlljZ3UzxmpsvKVm0huUJpUzfpMaH84qYlXDp3+vd0WpwZza7Ktgl/4yt3lmqWZcdS295LeUsPqdFn35ofMTctisduX05z94BOya1Oy+8S/VjZI6Mfmz4Y/QhQ1aqtI1cobeqhICWSqxelTbsbsSezODOGjr6hCS/nNzIb5pr8RBzGmlI7NWZyPZHmpVlTSmjjRJ3O9P/fNwkJEUFEBgdQ2tRNcUMn+c4WvtY8J8/hMJQ1dfvUsneLM62RvTsnWL4pb+4hyG4b7fs+MOwgbRIterBu0CZHBWvjRJ2WXyd6ESE7IZzSpm5KGrpYmBFDfHgQVdo6mrT6zj56B4d9KtHnJUUQHmQ/6xGpQ8MOKlt6KG/uJiMu9Jh/k8mUbkZkxoZp40Sd1qQSvYj8p4jsE5G9IrJBREJEJEdEtohIiYg8ISKun8DbhbITwtle3kp9Rz8LM6LJiAujoqWHTYca+P4L+zXpT1Cpc36YXB9K9HabsHZ2Ev/YXUvf4JnnTGrtHqBvcJhH3i5l7f9tYltZKzPiwkiMDCY00JqmebKlG7D611e19tLQ2afTGKuTmnCiF5F04AtAoTFmPmAHbgTuBX5mjMkDWoE7XBGou+QkhNMzMEygXbhqYSoZsaG8U9LMbb/fxiNvl/LwWzpL4EQcddaxcxJ9J9ED3Lwii/beQV7cU3vGfT/64Lt8/i87eLKokiGHoamrnxnx4YgIWXHWpGlp0ZNP9JlxYdS293L7Y9v48pM7J3085XsmW7oJAEJFJAAIA2qBi4CnnK+vB66d5DncKifB+g93yZxk4iOCWZUbT3pMKD++fiGXzk3mhd01Osx8AkoauggJtJE8gT7i3mxVbjy5CeH8ZUvFaferbuvlaFM3rx+o50hjN7Fh1syUI7NiZjn/To2Z/L9PRmwoDgN7qzt0nWR1UhNO9MaYauD/gAqsBN8ObAfajDFDzt2qgPSTvV9E7hSRIhEpamz03OpPC9JjCLAJn1w5A4BPrJzBO/dcxA2FmXx0aTpNXQP86o0Stpa2eCzG6ebht47yh81lLMuOwzZN5p8fLxHhpuVZFJW3crj+1GsSFzlX1AoNtBMUYOPnNy7BJjDfuYh9QXIk0aGBxLtgacLM2LDRx3UdfeMqKyn/MpnSTSxwDZADpAHhwOXjfb8x5iFjTKExpjAx0XNDuPOSItj93cs4Ny/hhNcuLEgiMiSAn79ezG2/30pDZ58HIpxe2noGuPflg1xYkMRvPnGOp8Nxi4+ek0GQ3cZDbx7lH7trGRp28PWndnHP07tH9ykqayUiOIBHbivkJ85Beju+fRnLsq0eN59bO5MX7l59ypkqz0bGmERvjHYPVieaTOnmEqDUGNNojBkEngHOA2KcpRyADKB6kjG6XVjQyaf8CQm08+RnVvGbT5zDwJCDX24smeLIpp9/7KllcNjw5UtnER7sm1MpxYUHccWCFJ7aXsVdf3mfDVsreHZnDU+/X0VbjzWqeltZC0uyYjh3ZgLXLLa+1EY7yzdg/cxlxoWd9PhnKzUmhKAAG7NTrNHH2gNHHW8yib4CWCkiYWI1Sy4G9gNvANc797kVeHZyIXrWnNQoLp+fwr8ty2TD1gr2VLXjcBidD+cUnt1Rw8zE8NGBPL7qPy+ZxadW55AaHcL9G4vpH3IwOGz4xcYSbnroPQ7WdY6uX+xugXYbGz69kl/etARwzwIpanqbTI1+C9ZN1/eBPc5jPQR8A/iyiJQA8cAjLojT4762roCEiGDuWL+Ned95hT/o+rMnaO7qZ2tZC1cvSndJScKbZSeE899XzeXy+Sk0dQ0QaBfSY0J59J1SShq7uHXVDG5enjVl8ZwzI5a8pAhCA+2a6NUJJtXrxhjzHWPMbGPMfGPMJ40x/caYo8aY5caYPGPMx4wxPtGxNyYsiJ/+2yK6+oew24TndtV4OiSvM5Jg5qf7dmt+rMvmWguFFM6I4841ucyID+OJO1fyvWvmkxQ1tT2ORrptaqJXx/PrkbFn69yZCez73jr+/bxsdlS0unS1IV9Q7ZxBMc0Fg4Cmi2XZsSzKiOYjS9O59dxsNn31QnITPTclc2acjpJVJ9JEf5ZEhAsLrEmp3i7RhcbHqnb29kifZksGTkaA3cazn1/NxwozATxeshpZ0KR/SLtYqg9oop+ARRkxRIUE8Nr+OgDtt+xU3dZLZEgAUSGBZ95ZucXqvAT6Bh28W6LTFqsPaKKfgAC7jevPyeTvO2u48w9FLPzuq2wv1wFVNW29pPtR2cYbnZsXT2RwAC/tPfMUDcp/aKKfoK+tK2BmYjiv7q9HBL79931+P1VCVasmek8LDrBz8ZwkXttf7/c/j+oDmugnKDTIzvrbl/O7Wwr56Q2L2V/bwd0bdowOmPFH1W29flWf91aXz0+htWeQbWVnN52y8l2a6CchIzaMS+cmc+WCFL5+eQGv7a/nx68c8nRYHtHRN0hn35C26L3A6vxEAu3CG4caPB2K8hKa6F1ARPjchXmszI1nX3W7p8OZckcbu3jgDWt6CG3Re15EcAArcuL550FN9Mqiid6F8pIiKG7o8rvpER5+u5Tf/usocOwEW8pz1s5OoqShS/vUK0ATvUvlJ0fQMzBMTXsfzV39fPmJncd8fd5f08HF922ipdu36vgVzT3kJITzP9fMY6FzGl7lWWsLrBlhNx323BTgyntooneh/CRr9sDi+k5+vekIz+yo5t9/v40NW61FKjYfbeZIYzd7fay8U9HSw/z0aG5Zle1z889PVzkJ4SRGBo/Oi6/8myZ6F8pPsoa+v3ukmT+9V861i9OYkxrFk0WVAJQ3W8vr+cpcJMYYBocdVLf1MsNFU+4q1xARCmfEUqQ9bxSa6F0qNjyIhIggHn7rKMMOw39eOosPLUhhR0Ub9R19lDVbCd4XEv3QsIPV977BvS8dZNhhRtdAVd7jnBmxVLf1UteuC+b4O030LpaXFIHDwHeunseM+HDWzbNmN3x1f/0HLfrm6Z/ojzR2U93Wy5+2WNM1j6yBqrxHoXM1qyIdte33NNG72B2rc/nG5bP5xAprLvK8pAhyE8J5cXft6BJv5Sdp0Xf3D52wzZvtrmoDoG/QGn05QxO915mXFkVIoE3LN0oTvatdOjeZz144c3QWQxHhsnkpbD7azLDDEBUSQGVLzzFdMA/UdrDoe6+yq7LNU2GPy/byVl52zqGyZ8wN5aAAG8mRUzv3ujqzQLuN5TnxbDrU4HddftWxNNFPgcvmJY8+Xp2fQFf/0DFdLDceqGfIYdhX0+GJ8Mbt568f5lt/2wtYiX5RZgzBATYyY0O1t42XunROEmXNPRxp7PZ0KMqDNNFPgcUZMSRFBgNwfr7Vv3nsDdl3nFPKVrd5d+2+uL6L5u4B6tr72F/TwbIZsVy7OJ0LZiV5OjR1ChfNsRoZrx+o93AkypM00U8Bm024amEaCRHBowtG//S1wxxp7KJvcJjtFVYNdaSG743aewep67B6bzy3q5r+IQcLMqK59/qF/L8Pz/VwdOpU0mNCmZsaxT8P6HQI/kwT/RT5+uUFvPiF1cxMjODjK7LYXt7KD/5xgKKyVgaGHAQF2EZXaBrRNzjMbb/f6hUDrEoaukYfr3/X6mmzIifeU+Gos7A8J469Ne1ap/djAZ4OwF+EBNoJCbQD8IPrFuAw8MLuGnISwgm0CxfPTmJHxbE3Y8ube9h0qJHCGbHM9/DUAsX1nQAE2ITqtl7ykyJIidYbsNPBzCRrao66jj5So3XSOX80qRa9iMSIyFMiclBEDojIKhGJE5HXRKTY+Xesq4L1JSty4ujsG2LD1gpWzUxgVnIk9Z19DAw56Oofoqypm8bOfsA7BlgVN3QREmhjqbP0NHKvQXm/mYnhABxp0Buy/mqypZv7gZeNMbOBRcAB4B5gozEmH9jofK6OszzHGszSMzDMpXOTyYgNxRiobe/lFxuLufbX79DQadXEK1s8X7s/XN9JXlIEc1OjADg/P8HDEanxyku0puY40th1hj2Vr5pwoheRaGAN8AiAMWbAGNMGXAOsd+62Hrh2skH6orSYUDLjrK/Rl85JHp3Hvaq1l12VbbT1DHKoziqXeKpF39DRx1ee3MWDm46wtbSFualRXFiQyOyUSFbkxnkkJnX2EiODiQwO0ETvxyZTo88BGoHfi8giYDvwRSDZGDOyMnEdkHyyN4vIncCdAFlZWZMIY/q6bnE6h+u7SIkOYdC5vmdVaw8HnQl+h3MAVW17L4PDDgLtU3vv/PUDDTz9fhUACzOi+eq6ApIiQ7iwQLtTTiciQm5ShCZ6PzaZRB8ALAXuNsZsEZH7Oa5MY4wxInLSW/3GmIeAhwAKCwv9sjvAly8rGH2cEh1CgE14q7iJ9t5BAPZUWb1tHAZq2/rIig/jnwfr+elrh7lifip3rM4ZvcHrDmXN3QTZbTx2+zKWZMYSGuS+cyn3mpkYzrvO8RrK/0ymiVgFVBljtjifP4WV+OtFJBXA+bd24B2HQLuNVTPj+cee2tFtvYPDBDhHnFa09PBkUSW3P1ZEXXs/P3nlEL9+o4SntlfxkV+/Q32H62coLGvqJis+jHNnJmiSn+bykiKo6+hjp5dPs6HcY8KJ3hhTB1SKyEiz9GJgP/AccKtz263As5OK0I9cuSCVka7OQc4yzbw06+bnW8WNfOfZfZw7M563v7GWwhmxvFncxONbK3i/oo1PPrKFvsFhl8ZT1txNdny4S4+pPOP6pRlkxoXyyUe26PKCfmiyRd+7gT+LyG5gMfBD4EfApSJSDFzifK7G4bK5ydjEGs04MhvkgoxoAu3Cb988SlCAjZ/esJiQQDsrc+PZU93Ozso2CpIjOVzfNe5J0UZ685xMZ98gww6Dw2Eob+4hJ0FnpfSz6GEuAAAXe0lEQVQFSVEhPHLrMjr7hnizWJcX9DeTSvTGmJ3GmEJjzEJjzLXGmFZjTLMx5mJjTL4x5hJjjE6GPU7xEcFcMT+VNbMSR3vhpEaHcsGsRM7Li+ev/7FqdJDSytx4hh2GIYfh9tXZgDXA6kxe21/Pyh9upLTpxD7Vrd0DrL73DX7++mHqOvroH3IwQ1v0PiM/KYKI4AAOO2/2K/+hUyB4mQc+vpT//cgC0mOsRJ8YEczDty7jz59ayazkyNH9zpkRS6BdCAm0cfWidAJsQmnzB8m7tKmb+18vxuE49j73K/vqcBjYV3PitAqPvVtGe+8gj71bNjqTZk6CJnpfISLMSo4Y7dWl/Icmei810qJPjAo+6euhQXZW5yVw0ewkQoPsZMWFja5gBfD9F/bzs9cPs6Pyg0UnjDH867D1tX3s3DVgLXzy2LtlzEqOoLNviPtePQTogiK+piAlkkP1nTrvjZ/RRO+lRm6CjrTsT+ahWwq5/8YlgJWQS5us0s3e6nb+edDq7PTinjoA3i1p4rdvHh2dVuH4RL+1rIX23kG+fdVcLixI5GBdJ2FBdp0bxccUJEfS1jM4+nOg/INOaual1s1LYcOnjy3XHG/sAKrshHC2lLZgjOH375QRGRLA3NQoXtpTyxcuzuczf9pOZ5+1XOH89ChKGrqo7+jjzcONpMeGUlJvJf75adH87pZCdle1Y7cJdl1QxKfMSrF+ng7WdZIUpZPS+QtN9F7KbhNWzRz/NMA5CeH0DAzT2NnPtrIWVuclcMmcZL7y11185o9FdPYN8bV1BYQH2alu62X95nI+/Mu3aejsJz48iLWzk0iICCY2PAhgdN585VsKkkcSfQdrZunEdP5CSzc+YqR3zPbyVipaeliSFcOVC1I5Ly+e9462cPHsJO5am8dt5+WQlxTBwJCDhs5+rl6URnP3AG8cbGBWcoSHr0K5W3xEMAXJkfxjd+2Zd1Y+QxO9j8h19o7543vWoiBLsqwpC/54+woeuNnqyTMiL8lK6AszovnCxfkANHcPnLZMpHzHzSuy2FXVPjrFhvJ9muh9REZsKIszY3j3SDN2mzA/zVqoxGYTPrQw9Zh67OyUKOakRvGVywqYmRhOnLNck68ter9w3dJ0QgPt/GVruadDUVNEE72PEBH+44JcAOakRp52bprw4ABe+uL5XDArERGh0FmP1xa9f4gKCWTt7ETeKm7ydChqimii9yGXzk1haVYMl81NOav3rZmVSGigXRO9H1maFUtVa+9pp8NQvkN73fgQu0145nPnnfX7bl6exWXzkokODXRDVMobLcmKAWBnRRuXzTu7hoGafrRFr7DZhKRI7VPtT+alWZPl7dBpi/2CJnql/FBIoJ25qVHsqGg9885q2tNEr5SfOmdGHEVlrTzydqmnQ1FupoleKT9190V5XDArke+/sF9b9j5OE71Sfio2PIgfX78QgM1HdT1ZX6aJXik/Fh8RTH5SBFuO6vpAvkwTvVJ+bnlOHNvLWxkadng6FOUmmuiV8nPLc+Lo6h/iQK2uPOWrNNEr5ecKs+MA2Fmlfep91aQTvYjYRWSHiLzgfJ4jIltEpEREnhCRoMmHqZRyl5SoEGwCDR06HYKvckWL/ovAgTHP7wV+ZozJA1qBO1xwDqWUm9htQlx4EE1durygr5pUoheRDOBDwMPO5wJcBDzl3GU9cO1kzqGUcr+EiGAaOwc8HYZyk8m26H8OfB0YuV0fD7QZY4acz6uA9EmeQynlZomRwTRqi95nTTjRi8hVQIMxZvsE33+niBSJSFFjY+NEw1BKuUBCRDBNnf0UlbXw/K4aT4ejXGwy0xSfB1wtIlcCIUAUcD8QIyIBzlZ9BlB9sjcbYx4CHgIoLCw0k4hDKTVJIy36X71Rwt7qdj68KM3TISkXmnCL3hjzTWNMhjEmG7gR+Kcx5uPAG8D1zt1uBZ6ddJRKKbdKiAhiYMjBrso2mroG6Bsc9nRIyoXc0Y/+G8CXRaQEq2b/iBvOoZRyocTIYABaewYBqGrt9WQ4ysVcssKUMWYTsMn5+Ciw3BXHVUpNjYSI4GOeV7f1kpeki8X7Ch0Zq5Q6IdFXtfZ4KBLlDrpmrFJqtHRjE7CJaOnGx2iiV0oRGxaETSArLgyD1uh9jSZ6pRR2mxAfEUxeUgS9g8NauvExWqNXSgHw/WvmcfdF+WTEhFGtLXqfoi16pRQAl89PBeDNw400dPbTOzBMaJDdw1EpV9BEr5Q6xqLMGAA2HWpga1kLV8xPZXlOnIejUpOhiV4pdYxzZ8aTGBnM/3tuH42d/TgcRhP9NKc1eqXUMQLsNq5dnEZjpzWbZUOnzmo53WmiV0qd4N+WZRIaaCcuPIh6XXlq2tNEr5Q6QV5SJPu+t441+QnaovcBmuiVUidlswlJUSE0dPZjjM4kPp1poldKnVJSZDADQw46eofOvLPyWprolVKnNDIHTkOn1umnM030SqlTSooMAbTnzXSniV4pdUrJUdqi9wWa6JVSp5QUZbXo6zu0RT+daaJXSp1SRHAAYUF2GjTRT2ua6JVSp5UUGcyuqja2lrZ4OhQ1QZrolVKnlZ0QzvbyVj7x8BZ6BrSb5XSkiV4pdVq/uGkJP7xuAQPDDnZWtHk6HDUBE070IpIpIm+IyH4R2SciX3RujxOR10Sk2Pl3rOvCVUpNtaiQQK5alIoIbC3T8s10NJkW/RDwFWPMXGAlcJeIzAXuATYaY/KBjc7nSqlpLCokkDkpUVqnn6YmnOiNMbXGmPedjzuBA0A6cA2w3rnbeuDayQaplPK85Tlx7KhoY3DY4elQ1FlySY1eRLKBJcAWINkYU+t8qQ5IdsU5lFKetWpmPL2Dw/x9R7WnQ1FnadKJXkQigKeBLxljOsa+Zqwp70467Z2I3CkiRSJS1NjYONkwlFJudsmcZJZlx/I/z++nps1aPHxgyMHOyjaGHTq7pTebVKIXkUCsJP9nY8wzzs31IpLqfD0VaDjZe40xDxljCo0xhYmJiZMJQyk1Bew24b6PLWbQ4eBHLx0EYP27ZVz7wDus+fEbHK7v9HCE6lQm0+tGgEeAA8aYn4556TngVufjW4FnJx6eUsqbZMWH8anVuTy3q4bdVW28dqCezLhQegaG+K9n9ui89V5qMi3684BPAheJyE7nnyuBHwGXikgxcInzuVLKR3zmglziw4P49rP72F7eytWL0vjmFXMoKm/l15uOaLL3QgETfaMx5m1ATvHyxRM9rlLKu0WGBPKlS/L59rP7AFhbkMTSrFjeONTAT145REffIN+8Ys4J7+vuH2J7eStrZmmpdqrpyFil1Fm7cXkWuQnhRIcGsjgzBptNeODmpVw2N5knt1XiGHNz9r//vocNWyv443vl3PLoVuradcrjqTbhFr1Syn8F2m08dEshrT0DBNit9qLNJqybl8Kr++v589YKHnyjhPtuWMyf3qtgUUY0qdGhAJQ2dZMSHeLJ8P2OJnql1ITkJUWcsO28vAQAvv/8fgaGHdy94X0A9tV0UONsyZc3d7NqZvy4z3O4vpO0mFAigjVdTZSWbpRSLpMSHUJuYjgDww7Cg+w0dQ0QaBeGHIZG53KE5S094z7esMNw3QPvcP/rh90Vsl/QRK+Ucqk1+YkE2W388uYlANyyKnv0NRGrRQ/w2v56Nh6oP+2xmrv66R4YnvQcO8/vqmFnpf/OvKmJXinlUl++bBbP3X0eF81O5unPruJr6wrITQhHBJbNiKO82WrR/88L+0YHXjV09HH3hh28X9F6zLFqneWefTUd9A4Mj+v8fy2q5Jn3q0aft/cO8pUnd/Hbfx0Z9zVsOdrMk9sqx72/t9Oil1LKpaJCAolKCQTgnBlxAFw6N5ldVW3MToniqe1V1Hf0UdnSi02gpq2Xm3/3HmXNPeyoaOXlL60ZrcfXtltTLQw5DLuq2liWHcdbxY30DAwz7DAsz4kjKTKYzv4hokIC6R8a5vsv7Cc9NoyPLM0A4OW9tQwMO2jqGv9yiA9sOsI7JU2cmxdPRmyYK/95PEITvVLK7b55pdWv/tG3S+nqH+KVfXUAOAz8+OWDlDX38K0r5/DDlw5w8X2b+MjSDL5x+ezRFj3A9vJW6jv6+OLjO0e3rciJ40MLU/l/z+4jLymCqxam0tE3hKOlB2MMIsLfnJOwNXUNjDveQ3UdDDsM698t41sfmuuKfwKP0tKNUmrKzIi3WsdPFlVicw63fH53LbkJ4Xx6TS6/umkpKVEhPPzWUfoGh6lr7yPIbiM3MZz3jjbz8t46kiKDeflL53P3RXlsKW3hJ68coiA5kq6+IX7+ejEAXf1DtPUMUt7czZbSFoICbDR1HtuiP9rYRVf/iUsjtvUMUN/RT3CAjQ1bK8ddMhrxTkkTDR3eNVZAE71SasosyowhKiSAvdUdrMiJJzI4gGGHGR0t+6GFqXzh4nwGhw07K9uobe8jJTqEqxel8VZxExsPNnDJ3GRmp0TxqfNzCQuy09k3xLevmsuDn1hKoF3ITQgHoKKlh9+9dZRAm42blmXS2T9Ee+8ge6raaeke4Ir73+Jjv9lMe8/gMTEerLMmZ/voORl09Q9x6DSTtd3x2DZ++toHPYKGhh38+++38eBZ3A+YCprolVJTJiEimL98eiVJkcFcsSCFOWlRAJyfnzC6T+GMOGvZwtIW6tr7SI0O4dZV2YQG2hkYcnDpXGuJi+jQQO5am8cV81M4Ly+eJVmx/Otra7n/Rqu3z66qNv5aVMVHlqYzJ9U6z683lfDhX73Nd5/bR/+Qg+L6Tr74xI7Rc3f1D3Gw1ppt/drF6YBVxgHoHxpmYOiDRVdq2nrZeLCBx7dWjI4EbuzqZ2DYwdHG7pNe/8G6DnYcd8N5KmiNXik1peanR/PeNy9GBCpbethZ2caK3A8GUEWHBTLbuWxhbUcv52TFEhsexC3nzuDp7VWcO2aw1V1r8445dlpMKFGh1o3gh948Sv+Qg9tX51Dh7Omz+UgzAM/tqmFeWhTXLk7nBy8eYFtZC1lxYVz0f5uwiRATFkjhjFjCguwcqLVa9J9aX0RIoJ37bljEHzeXExxgtZMbOvvZU93OosyY0XsKpU3dlDZ1s728levPsW4KP1lUybf+tof48GDe+6+pnQ5MW/RKqSlnswkiwufX5vP3z513wqjXFTlxFJW3UNnSS4pz6oSvr5vNpq+tJTjAftpjRwQHEBsWSFVrL9nxYcxKjiQhMhiwumnanTcHPro0g0+snEFCRDA/e+0wr+2vp3tgmM7+IQqSI7HZhFnJkRyq6+RgXQdvFTfx3tFm/rG7lp+8coh7Xz5IYmQwNoHXneMBRubxqWrt4f7XD/PVv+7iSGMXAD9++RCDw4a6jr6T3htwJ030SimPiQ4LZK6zfDPWhxel0jdolUlSoqwkbbfJuKdByIqzbvpeWJAEQHx4EGCNtL1ifgr337iYj6/MIjTIzmcvnMm7R5p5cNMRsuLCuHlFFjcUZgIwOyWSg3Ud/Om9cgA6+4Z4bmcNAIPD1rEKs+N45v1qGjr7Rlv0DgOv7LOS/5NFlXT2DdLU1c+ijGgAyppOXtpxF030Simvc86MOD534UwAMuPOvh97hvM9F822En2is0UPMDMxgmsWp49+M/j4iiySIoOpbuvl0rnJ/PC6BXzUWW6ZnRJJa88gT26rGq3zbz7azKrceP79vGxuWZXN19YV0NozwM2/20LlmOkdegeHCbAJT2+vorjBatWvdcZTqoleKaXga+sK+McXVrPW2So/G3NTo4gLD2J5jjVgKyTQPvptIMfZK2dESKB99JfKunkpx7w2ktxzE8N5+NZCApxlnxW5cXznw/PIS4pgWXYcP7huPiUNXby2v56EiKDR9992bjZNXQM8td0aqXuBs3eRtuiVUgoQEealRWOznWp9o1O7c00ub3zlQkICP6jnjyTg7OMSPVjz8Tx713mjvxhGLM+J41c3L+GJO1eRHhPKrORIABZnxhyz38isndVtveQnRZIQEUSgXbhzTS5gzbUDMDslitToEEqd8/08v6uGEmdr35000SulfE6g3UZ0WOAx2xIirPJNTvyJid5mExYdl7zB+mVz1cK00WMtSLdq7Mcn+qTIEDLjrJvGqdEhzE6JYklmLElRIcxKjqCzb4jU6BBCg+xkx4dT2tRN3+Aw//nEztHWvjtp90qllF9IjAwmNizwhF8AZ+PTa3JYkhVDTFjQCa8tzYp19hIKGZ3yAaz7DYfru0ZLRtkJ4by8t5a91e0MOQxLs078BeNq2qJXSvmFz1wwkx9ct2BSx8hLiuTG5VknfW1pVixgtegTI4NHbwAXzrC2j5SMZiaG09ozyEt7rfl+ljpfdydN9Eopv7A4M4YrF6S67firZsZjE8h31vFHLMu26v55idaKXCNdPv+wuYysuLDRkpI7uS3Ri8jlInJIREpE5B53nUcppbzBrORItvzXJazMPXaZxKz4MDZ8eiU3Lrf65uclRbAkK4bBYcOSKSjbgJsSvYjYgQeAK4C5wE0iMv3n+lRKqdMY219/rFUz4wkL+uCW6MfOsZL+SLnH3dx1M3Y5UGKMOQogIo8D1wD73XQ+pZSaNq5dksaRxi6uWui+UtJY7kr06cDYdbiqgBVuOpdSSk0rYUEBfPuqqStyeOxmrIjcKSJFIlLU2NjoqTCUUsrnuSvRVwOZY55nOLeNMsY8ZIwpNMYUJiYmuikMpZRS7kr024B8EckRkSDgRuA5N51LKaXUabilRm+MGRKRzwOvAHbgUWPMPnecSyml1Om5bQoEY8yLwIvuOr5SSqnx0ZGxSinl4zTRK6WUj9NEr5RSPk6MMZ6OARFpBMon8NYEoMnF4Xg7vWb/4Y/Xrdd8dmYYY87YP90rEv1EiUiRMabQ03FMJb1m/+GP163X7B5aulFKKR+niV4ppXzcdE/0D3k6AA/Qa/Yf/njdes1uMK1r9Eoppc5surfolVJKncG0TfT+slShiJSJyB4R2SkiRc5tcSLymogUO/+emmVq3EREHhWRBhHZO2bbSa9RLL9wfu67RWSp5yKfuFNc83dFpNr5We8UkSvHvPZN5zUfEpF1nol6ckQkU0TeEJH9IrJPRL7o3O6zn/VprnlqP2tjzLT7gzVR2hEgFwgCdgFzPR2Xm661DEg4btuPgXucj+8B7vV0nJO8xjXAUmDvma4RuBJ4CRBgJbDF0/G78Jq/C3z1JPvOdf6MBwM5zp99u6evYQLXnAosdT6OBA47r81nP+vTXPOUftbTtUU/ulShMWYAGFmq0F9cA6x3Pl4PXOvBWCbNGPMm0HLc5lNd4zXAH4zlPSBGRKZmPTYXOsU1n8o1wOPGmH5jTClQgvV/YFoxxtQaY953Pu4EDmCtRuezn/VprvlU3PJZT9dEf7KlCk/3jzedGeBVEdkuInc6tyUbY2qdj+uAZM+E5lanukZf/+w/7yxTPDqmJOdz1ywi2cASYAt+8lkfd80whZ/1dE30/mS1MWYpcAVwl4isGfuisb7v+XTXKX+4RqcHgZnAYqAWuM+z4biHiEQATwNfMsZ0jH3NVz/rk1zzlH7W0zXRn3GpQl9hjKl2/t0A/A3ra1z9yFdY598NnovQbU51jT772Rtj6o0xw8YYB/A7PvjK7jPXLCKBWAnvz8aYZ5ybffqzPtk1T/VnPV0TvV8sVSgi4SISOfIYuAzYi3Wttzp3uxV41jMRutWprvE54BZnj4yVQPuYr/3T2nH15+uwPmuwrvlGEQkWkRwgH9g61fFNlogI8AhwwBjz0zEv+exnfaprnvLP2tN3pSdxN/tKrDvYR4BveToeN11jLtYd+F3AvpHrBOKBjUAx8DoQ5+lYJ3mdG7C+vg5i1STvONU1YvXAeMD5ue8BCj0dvwuv+Y/Oa9rt/A+fOmb/bzmv+RBwhafjn+A1r8Yqy+wGdjr/XOnLn/VprnlKP2sdGauUUj5uupZulFJKjZMmeqWU8nGa6JVSysdpoldKKR+niV4ppXycJnqllPJxmuiVUsrHaaJXSikf9/8BlfyrZ5r5j0cAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x, y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Init and Train Linear Regression Model\n",
    "\n",
    "> ☝🏻This is the place where you might want to play with model configuration.\n",
    "\n",
    "- `polynomial_degree` - this parameter will allow you to add additional polynomial features of certain degree. More features - more curved the line will be.\n",
    "- `num_iterations` - this is the number of iterations that gradient descent algorithm will use to find the minimum of a cost function. Low numbers may prevent gradient descent from reaching the minimum. High numbers will make the algorithm work longer without improving its accuracy.\n",
    "- `learning_rate` - this is the size of the gradient descent step. Small learning step will make algorithm work longer and will probably require more iterations to reach the minimum of the cost function. Big learning steps may couse missing the minimum and growth of the cost function value with new iterations.\n",
    "- `regularization_param` - parameter that will fight overfitting. The higher the parameter, the simplier is the model will be.\n",
    "- `polynomial_degree` - the degree of additional polynomial features (`x1^2 * x2, x1^2 * x2^2, ...`). This will allow you to curve the predictions.\n",
    "- `sinusoid_degree` - the degree of sinusoid parameter multipliers of additional features (`sin(x), sin(2*x), ...`). This will allow you to curve the predictions by adding sinusoidal component to the prediction curve."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initial cost: 155308772.91\n",
      "Optimized cost: 1290750.59\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model Parameters</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>97.116343</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.247582</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-39.357423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-19.989335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8.242287</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>50.461720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>-44.046953</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>49.305197</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>-16.085046</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>25.125076</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>-32.604765</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>-25.009579</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>-24.051643</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>-30.740310</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>-54.837941</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Model Parameters\n",
       "0          97.116343\n",
       "1           4.247582\n",
       "2         -39.357423\n",
       "3         -19.989335\n",
       "4           8.242287\n",
       "5          50.461720\n",
       "6         -44.046953\n",
       "7          49.305197\n",
       "8         -16.085046\n",
       "9          25.125076\n",
       "10        -32.604765\n",
       "11        -25.009579\n",
       "12        -24.051643\n",
       "13        -30.740310\n",
       "14        -54.837941"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Set up linear regression parameters.\n",
    "num_iterations = 50000  # Number of gradient descent iterations.\n",
    "regularization_param = 0  # Helps to fight model overfitting.\n",
    "learning_rate = 0.01  # The size of the gradient descent step.\n",
    "polynomial_degree = 4  # The degree of additional polynomial features.\n",
    "sinusoid_degree = 3  # The degree of sinusoid parameter multipliers of additional features.\n",
    "\n",
    "# Init linear regression instance.\n",
    "linear_regression = LinearRegression(x, y, polynomial_degree, sinusoid_degree)\n",
    "\n",
    "# Train linear regression.\n",
    "(theta, cost_history) = linear_regression.train(\n",
    "    learning_rate,\n",
    "    regularization_param,\n",
    "    num_iterations\n",
    ")\n",
    "\n",
    "# Print training results.\n",
    "print('Initial cost: {:.2f}'.format(cost_history[0]))\n",
    "print('Optimized cost: {:.2f}'.format(cost_history[-1]))\n",
    "\n",
    "# Print model parameters\n",
    "theta_table = pd.DataFrame({'Model Parameters': theta.flatten()})\n",
    "theta_table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze Gradient Descent Progress\n",
    "\n",
    "The plot below illustrates how the cost function value changes over each iteration. You should see it decreasing. \n",
    "\n",
    "In case if cost function value increases it may mean that gradient descent missed the cost function minimum and with each step it goes further away from it. In this case you might want to reduce the learning rate parameter (the size of the gradient step).\n",
    "\n",
    "From this plot you may also get an understanding of how many iterations you need to get an optimal value of the cost function. In current example you may see that there is no much sense to increase the number of gradient descent iterations over 500 since it will not reduce cost function significantly.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xm4XVV9//H359whCUlIgFwZkmBQQQQZvSAqVmptDUiJWhUoDiiYWodfW+kAapHi0xaljz+1ojQ/pKhVKEXUPJiKtEKxIMNFIEwGw6AJU66ZSAIZbu7398de52TfmzPcDDvn3uzP63nOc/Zee+291zr33PM9a6191lZEYGZmBlBpdwHMzGz0cFAwM7MaBwUzM6txUDAzsxoHBTMzq3FQMDOzGgcFawtJT0p6S1r+lKQr2l0mM3NQsDoknSHpTknrJC1Lyx+VpCLOFxH/EBHn7uhxJM2SFJI6m+S5SNImSWvS41FJX5W0/46evyipTq9osv1sSZslrZX0vKT7JJ26K8touw8HBRtC0nnAl4FLgf2AfYGPAG8Auhvs07HLCrhz/HtETAb2Bt5BVs97RnNgGIGfR8QkYCrwDeBaSXsNz9QsYG6PnX08az8HBauRNAW4GPhoRFwXEWsic29EnBURG1K+qyR9XdICSeuA35X0Nkn3pm+qSyRdNOzY75P0a0nLJX162LaLJP1bbv0ESbdLWiXpfkkn5bbdIulzkm5L3/R/Imla2nxrel6VvjW/rll9I2JTRDwEnA70A+flznNq+sa9KpXlyNy2v5H0VDr/Ikm/l9I7UlfYY2nbPZJmpm2HSrpJ0oq0z3tyx7tK0mWSfpT2u1PSy9O2ap3uT3U6vUWdBoErgQnAyyWdJGlpKvOzwL+m435Y0uJUnvmSDsiV5w9SGVdL+pqk/5F0btp2dnrt/6+k5cBFKf1Dkh6RtFLSjZJemtKV8i5L740HJL06bTtF0sOpzk9J+stmdbNdJCL88IOIAJgNDACdLfJdBawmaz1UgPHAScARaf1I4Dng7Sn/YcBa4HeAccAX03nekrZfBPxbWp4OLAdOScf6/bTek7bfAjwGHEL2wXcLcEnaNguIZuXPn2tY+sXAnWn5GGAZ8FqgA/gA8GQq+yuBJcABuXO+PC3/FfBAyiPgKGAfYGLa54NAZzr+b4HDcq/ncuD4tP07wDW5sgXwiiZ1Ohv437TcCfwZsAaYkv4uA8DnU/knAG9O5z82pf0zcGvafxrwPPDO3LE2AefmzjUAfCJtnwDMARYDr0ppnwFuT/nfCtxD1oJRyrN/2vYM8Ma0vBdwbLv/B/yIsdlSkHRl+ubx4AjyHijp5vQtdqGkU3ZFGceoacBvI2KgmpD7xv6ipN/J5f1hRNwWEYMRsT4ibomIB9L6QuBq4E0p77uAGyLi1shaG38LDDYow3uBBRGxIB3rJqCPLEhU/WtEPBoRLwLXAkfvhLo/TdadBDAX+JeIuDMiNkfEN4ENwAnAZrIP0sMkdUXEkxHxWNrvXOAzEbEoMvdHxHLgVODJiPjXiBiIiHuB7wHvzp3/+xFxV3rtv7MddTpB0irgWeBM4B0RsTptGwQ+GxEb0mt2FnBlRPwi/T0uAF4naRbZ6/xQRFyfyvKVdMwhr1VE/HOqy4tk3Yv/GBGPpH3+ATg6tRY2AZOBQwGlPM+k42xKr+OeEbEyIn6xjXW2AozJoED2zWr2CPN+Brg2Io4BzgC+VlShdgPLgWn5fuKIeH1ETE3b8u+XJfkdJb02Bd9+SavJPiiq3ToH5PNHxLp0vHpeCrw7BaJV6YPuRCDf35//kHoBmLQtlWxgOrAiV4bzhpVhJlnrYDHw52QtjmWSrsl1vcwka8XUq9Nrhx3vLLKxjJ1VpzsiYmpETIuIEyLiv3Lb+iNifW79AODX1ZWIWEv295jO1n+rAJYOO9eSYesvBb6cq9sKslbB9Ij4KfBV4DKy12uepD3Tfn9EFoR+nbqomnb32a4xJoNCRNzKln9gACS9XNKPUz/uzyQdWs0OVN+EU8i+EVp9Pyf7RjxnBHmHT6/7XWA+MDMipgCXk30wQNZNMLOaUdIeZN0q9SwBvp0+4KqPiRFxyXaUaUQkVYA/BH6WK8PfDyvDHhFxNUBEfDciTiT7MAyyrpnqfi9vUKf/GXa8SRHxp9tT3u0w/HV5mqzsAEiaSPb3eIrsbzUjt0359QbHWwL8ybD6TYiI2wEi4isR8RqybsRDyLrZiIi7I2IO8BLgB2StPmuzMRkUGpgHfCK9+f6SLS2Ci4D3SloKLCDrC7U6ImIV8HfA1yS9S9JkSRVJR5P1izczGVgREeslHQ/8cW7bdcCpkk6U1E3Wf9/ovfdvwB9KemsauB2fBkuHfzDV00/WVfKyEeRFUqekV5F1de1HNtYB8P+Aj6TWjyRNVDaQPlnSKyW9WdI4YD3wIlu6wq4APifp4LTfkZL2AW4ADlE22N6VHselc4/EcyOt0whdDXxQ0tGpHv9ANp7yJPAj4AhJb08txo8xtEVTz+XABZIOh+yCBUnvTsvHpdexC1hH9poNSuqWdJakKRGxiWwco1GXou1Cu0VQkDQJeD3wH5LuA/6FLd0NZwJXRcQMsqbqt9M3Q6sjIr4AfBL4a7IPo+fIXs+/AW5vsutHgYslrQEuJPetL7IrfD5G1pp4BljJ1l0S1bxLyFoqnyL7kF9C9s2y5d8sIl4A/h64LXVlnNAg6+mS1pINls8n6zp5TUQ8nY7TB3yYrNtjJdkg6tlp33HAJWQDtc+Sfcu9IG37Yqr3T8g+5L4BTIiINcAfkHVfPp32qw78jsRFwDdTnd7TKnMrqWvpb8nGNZ4ha92ckbb9lmys4wtkr8thZGM6G5oc7/tk9blG0vPAg8DJafOeZEF2JVmX1XKyy50B3gc8mfb5CFmXmrWZsi7DsScNit0QEa9OfZSLImKr68wlPQTMTh82SHocOCEilu3K8pqNRekL1FLgrIi4ud3lseLtFt+YI+J54Ilck1WSjkqbfwNUryN/Fdnlk/1tKajZGJC67qamrqVPkY0N3dHmYtkuMiaDgqSryQZFX6nshznnkDU9z5F0P/AQWwZLzwM+nNKvBs6Osdo8Mts1Xkd2FdVvyQbg354uPbUSGLPdR2ZmtvONyZaCmZkVY8xNZjVt2rSYNWtWu4thZjam3HPPPb+NiJ5W+QoLCpKuJPt5/7KIeHWDPCcBXwK6yKZXeFO9fHmzZs2ir69vZxbVzGy3J+nXrXMV2310FU2mopA0lewHZqdFxOEMnQfGzMzaoLCgUG8qimH+GLg+In6T8vt3A2ZmbdbOgeZDgL2UzY9/j6T3N8ooaa6kPkl9/f3+iYGZWVHaGRQ6gdcAbyObc/1vJR1SL2NEzIuI3ojo7elpOU5iZmbbqZ1XHy0FlqdplNcpu8PUUcCjbSyTmVmptbOl8EPgxDRT5R5kd7l6pI3lMTMrvSIvSb2a7FaA09K01Z8lu/SUiLg8Ih6R9GNgIdmUuVdERMs7qZmZWXEKCwoRceYI8lzKlml0C/Xoc2u44f6nef/rZzFt0khnLDYzK5fSTHPxq+fW8pWfLmb52o3tLoqZ2ahVmqAgtc5jZlZ2pQkKVbF9t/E1MyuF0gSFakPBM4WbmTVWnqDg7iMzs5ZKExSq3FIwM2usREEhayp4TMHMrLHSBAV3H5mZtVaaoFDl7iMzs8ZKExTcUDAza608QcH9R2ZmLZUmKFS5+8jMrLHSBIXaj9d89ZGZWUPlCQruPTIza6k0QaHK3UdmZo2VJihUWwqOCWZmjZUnKPiiVDOzlgoLCpKulLRMUtNbbEo6TtKApHcVVZa8cP+RmVlDRbYUrgJmN8sgqQP4PPCTAsuRTpY9OSSYmTVWWFCIiFuBFS2yfQL4HrCsqHJUufPIzKy1to0pSJoOvAP4+gjyzpXUJ6mvv79/h87r3iMzs8baOdD8JeBvImKwVcaImBcRvRHR29PTs10n2zLNhaOCmVkjnW08dy9wTfqwngacImkgIn5QxMncfWRm1lrbgkJEHFRdlnQVcENRAWHoeYs+g5nZ2FVYUJB0NXASME3SUuCzQBdARFxe1Hkblyd7dkwwM2ussKAQEWduQ96ziypHlX+8ZmbWWml+0Vzl7iMzs8ZKExRq3UeOCmZmDZUnKLS7AGZmY0BpgkKV2wlmZo2VJyjUuo/aWwwzs9GsNEHBVx+ZmbVWmqBQ5Xs0m5k1Vpqg4KmPzMxaK09QaHcBzMzGgNIEhSo3FMzMGitNUKhOne2rj8zMGitRUGh3CczMRr/SBIUqX31kZtZYaYJC7eIjxwQzs4bKExTcfWRm1lJpgkKVGwpmZo0VFhQkXSlpmaQHG2w/S9JCSQ9Iul3SUUWVJZ0R8NTZZmbNFNlSuAqY3WT7E8CbIuII4HPAvALL4u4jM7MRKPJ2nLdKmtVk++251TuAGUWVZch5d8VJzMzGqNEypnAO8J9FnqDWUHBUMDNrqLCWwkhJ+l2yoHBikzxzgbkABx544PaeZ7v2MzMrk7a2FCQdCVwBzImI5Y3yRcS8iOiNiN6enp4dOqd/vGZm1ljbgoKkA4HrgfdFxKOFny89++IjM7PGCus+knQ1cBIwTdJS4LNAF0BEXA5cCOwDfC117QxERG9x5SnqyGZmu48irz46s8X2c4Fzizp/4/Pu6jOamY0do+Xqo8JV79HsmGBm1lh5goK7j8zMWipNUKjyNBdmZo2VLyi0uwBmZqNYaYKCu4/MzForTVCocu+RmVljpQkK2vLztbaWw8xsNCtPUHD3kZlZS6UJClXuPjIza6w0QaHaUnBMMDNrrDxBAfcfmZm1UpqgUOXuIzOzxkoTFLZ0HzkqmJk1Up6g0O4CmJmNAaUJClXuPjIza6w0QcFXH5mZtVaaoOAOJDOz1goLCpKulLRM0oMNtkvSVyQtlrRQ0rFFlSXPU2ebmTVWZEvhKmB2k+0nAwenx1zg6wWWxdNcmJmNQGFBISJuBVY0yTIH+FZk7gCmStq/qPI4JpiZtdbOMYXpwJLc+tKUthVJcyX1Serr7+/foZO698jMrLExMdAcEfMiojcient6erbrGEr9R/7xmplZY+0MCk8BM3PrM1JaIdx9ZGbWWjuDwnzg/ekqpBOA1RHxTNEndfeRmVljnUUdWNLVwEnANElLgc8CXQARcTmwADgFWAy8AHywqLJk5cmeHRTMzBorLChExJkttgfwsaLOP5ynzjYza21MDDTvTG4omJk1VpqgsKX7yGHBzKyR0gQFMzNrrXRBwe0EM7PGShMUanMfOSqYmTVUoqDgq4/MzFopTVCo8jQXZmaNlSYo1HqPHBPMzBoqT1Bw75GZWUulCQpVbiiYmTVWmqBQnebC3UdmZo2VJihUUvfRoKOCmVlD5QkKlWpLwUHBzKyR8gSFNNK8edBBwcyskREFBUnfHknaaNaRgoJjgplZYyNtKRyeX5HUAbxm5xenOEo19ZiCmVljTYOCpAskrQGOlPR8eqwBlgE/3CUl3EkqtZaCg4KZWSNNg0JE/GNETAYujYg902NyROwTERe0Orik2ZIWSVos6fw62w+UdLOkeyUtlHTKDtSlKXcfmZm1NtLuoxskTQSQ9F5JX5T00mY7pC6my4CTgcOAMyUdNizbZ4BrI+IY4Azga9tU+m1Q/UWzB5rNzBobaVD4OvCCpKOA84DHgG+12Od4YHFEPB4RG4FrgDnD8gSwZ1qeAjw9wvJssw5fkmpm1tJIg8JAZJ+mc4CvRsRlwOQW+0wHluTWl6a0vIuA90paCiwAPlHvQJLmSuqT1Nff3z/CIg9VcfeRmVlLIw0KayRdALwP+JGkCtC1E85/JnBVRMwATgG+nY49RETMi4jeiOjt6enZrhNV3H1kZtbSSIPC6cAG4EMR8SwwA7i0xT5PATNz6zNSWt45wLUAEfFzYDwwbYRl2iaSkNx9ZGbWzIiCQgoE3wGmSDoVWB8RrcYU7gYOlnSQpG6ygeT5w/L8Bvg9AEmvIgsK29c/NAIVic0OCmZmDY30F83vAe4C3g28B7hT0rua7RMRA8DHgRuBR8iuMnpI0sWSTkvZzgM+LOl+4Grg7Cjwq3yH5DEFM7MmOkeY79PAcRGxDEBSD/BfwHXNdoqIBWQDyPm0C3PLDwNv2JYC7wjJP14zM2tmpGMKlWpASJZvw76jRkVi0E0FM7OGRtpS+LGkG8m6eCAbeF7QJP+o1FFx95GZWTNNg4KkVwD7RsRfSXoncGLa9HOygecxRfIlqWZmzbRqKXwJuAAgIq4HrgeQdETa9oeFlm4n66jIl6SamTXRalxg34h4YHhiSptVSIkKVPHVR2ZmTbUKClObbJuwMwuyK1SEf6dgZtZEq6DQJ+nDwxMlnQvcU0yRilORu4/MzJppNabw58D3JZ3FliDQC3QD7yiyYEWoSB5oNjNromlQiIjngNdL+l3g1Sn5RxHx08JLVgBfkmpm1tyIfqcQETcDNxdclsJJ+MdrZmZNjLlfJe+I7OojBwUzs0ZKFRTcfWRm1lypgoJ8SaqZWVOlCgodnhDPzKypcgWFihhwUDAza6hUQaG7s8LA5sF2F8PMbNQqNChImi1pkaTFks5vkOc9kh6W9JCk7xZZns6K2LTZLQUzs0ZGej+FbSapA7gM+H1gKXC3pPnpbmvVPAeTzcL6hohYKeklRZUHoKujwia3FMzMGiqypXA8sDgiHo+IjcA1wJxheT4MXBYRKwGG3d1tp3NQMDNrrsigMB1YkltfmtLyDgEOkXSbpDskzS6wPHR1eKDZzKyZwrqPtuH8BwMnATOAWyUdERGr8pkkzQXmAhx44IHbf7KOChsH3FIwM2ukyJbCU8DM3PqMlJa3FJgfEZsi4gngUbIgMUREzIuI3ojo7enp2e4CdXdU3FIwM2uiyKBwN3CwpIMkdQNnAPOH5fkBWSsBSdPIupMeL6pAnR3ymIKZWROFBYWIGAA+DtwIPAJcGxEPSbpY0mkp243AckkPk83C+lcRsbyoMnV1VBjwJalmZg0VOqYQEQuABcPSLswtB/DJ9ChcV4fY6JaCmVlDpfpFc9ZScFAwM2ukVEGhs1LxL5rNzJooVVDo6vRAs5lZM+UKChX/otnMrJlSBYXOjuzOa5v9WwUzs7pKFRS6OrLqurVgZlZfyYKCAAcFM7NGShUUxnV2AHj+IzOzBkoVFMZ3ZdVd76BgZlZXyYJC1lJYv2lzm0tiZjY6lSooVLuPHBTMzOorVVCodR85KJiZ1VWqoDCh1n3kMQUzs3pKFRQ8pmBm1lxJg4JbCmZm9ZQsKGTVfdEtBTOzukoVFCa4+8jMrKlCg4Kk2ZIWSVos6fwm+f5IUkjqLbI84xwUzMyaKiwoSOoALgNOBg4DzpR0WJ18k4E/A+4sqixV1e6jDf5Fs5lZXUW2FI4HFkfE4xGxEbgGmFMn3+eAzwPrCywLAN0dFSR4caNbCmZm9RQZFKYDS3LrS1NajaRjgZkR8aNmB5I0V1KfpL7+/v7tLpAkxnd2uPvIzKyBtg00S6oAXwTOa5U3IuZFRG9E9Pb09OzQeSd0d7B+wEHBzKyeIoPCU8DM3PqMlFY1GXg1cIukJ4ETgPlFDzZP6OrghQ0OCmZm9RQZFO4GDpZ0kKRu4AxgfnVjRKyOiGkRMSsiZgF3AKdFRF+BZWLy+E7Wbhgo8hRmZmNWYUEhIgaAjwM3Ao8A10bEQ5IulnRaUedtZeI4BwUzs0Y6izx4RCwAFgxLu7BB3pOKLEvVpHGdrHph4644lZnZmFOqXzQDTBrfyRq3FMzM6ipfUOjuZJ2DgplZXeULCuM7WbveQcHMrJ7yBYVxnazbuJnBwWh3UczMRp3SBYXJ47Ox9XUb3VowMxuudEFh4rgsKPiyVDOzrZUuKEyqBgWPK5iZbaV8QWG8WwpmZo2ULijsmYLC824pmJltpXRBYeoe3QD+VbOZWR2lCwp7p6CwYp2DgpnZcKULCntO6EKClQ4KZmZbKV1Q6KiIqRO6WPnCpnYXxcxs1CldUADYa2I3KzymYGa2lXIGhT26PdBsZlZHaYPCinXuPjIzG66kQaHLA81mZnUUGhQkzZa0SNJiSefX2f5JSQ9LWijpvyW9tMjyVO0zaRwr1m0kwjOlmpnlFRYUJHUAlwEnA4cBZ0o6bFi2e4HeiDgSuA74QlHlydtvz3Fs3Dzo3yqYmQ1TZEvheGBxRDweERuBa4A5+QwRcXNEvJBW7wBmFFiemv2mTADgmdXrd8XpzMzGjCKDwnRgSW59aUpr5BzgP+ttkDRXUp+kvv7+/h0u2P5TxgPwrIOCmdkQo2KgWdJ7gV7g0nrbI2JeRPRGRG9PT88On68aFJ553kHBzCyvs8BjPwXMzK3PSGlDSHoL8GngTRGxocDy1OwzaRydFfHs6hd3xenMzMaMIlsKdwMHSzpIUjdwBjA/n0HSMcC/AKdFxLICyzJER0Xsu+d4nlnlloKZWV5hQSEiBoCPAzcCjwDXRsRDki6WdFrKdikwCfgPSfdJmt/gcDvdgXvvwRPL1+2q05mZjQlFdh8REQuABcPSLswtv6XI8zfzsp6J3LDwGSICSe0qhpnZqDIqBprb4eU9k1j94iaW+7cKZmY15Q0KL5kEwOP97kIyM6sqbVB4RQoKi55b0+aSmJmNHqUNCgdMGc+0Sd3c95tV7S6KmdmoUdqgIImjZ+7FvUtWtrsoZmajRmmDAsAxB07l8f51LF+7S34zZ2Y26pU6KLzx4GkA3Lxox+dTMjPbHZQ6KBwxfQr77Tmemx5+tt1FMTMbFUodFCRx8hH7cfMv+1m2xlNemJmVOigAvP91s9i4eZCrbnuy3UUxM2u70geFg6ZN5LSjDuCK/32Cx/rXtrs4ZmZtVfqgAPCZt72KPbo7OPebfSzzPRbMrMQcFICX7DmeK97fy7Or13PqP/8vP0oT5ZmZlY2DQtI7a2+u/+jr2XtiNx/77i9406W38MWfLOLOx5ezYWBzu4tnZrZLaKx9I+7t7Y2+vr7Cjr95MLhh4dNcc9cS7nhiORHQ3VnhFT2TOHT/ybxy38nM3HsPZuw1gRl77cFee3R56m0zG/Uk3RMRvS3zOSg0tvqFTdzxxHLu+fVKfvnsGhY9+zzPPT/01897dHfQM3kc+0zsZu+J6XlSN/tM7GbSuE4mjuusPU8c11FbntDVQXdnhc6KHFTMrHAjDQqF3mRH0mzgy0AHcEVEXDJs+zjgW8BrgOXA6RHxZJFl2hZT9ujirYfvx1sP36+WtvrFTSxd+QJPrXyRpenRv3YDK9ZtYOnKF1i4dBUr1m1kYHBkwVaCcZ0VujsqdHd2ZMtpfVxXha6OCh0V0SHR2SEqEp0VUakMfe6QsnzDHym9UhECKhIVZSeuKFsXZNsFIpeu7Lcc2X7UjiGptr0yJI+oVLJjKHeM2jm3OnaWlxQTq8dOxasdp7qNXJpqr9+WdamWM5c29HjU8tU5x7Djb4nVqnu8tGXo/mlleB1yRRt2DuXqsuX12KqOdco7/H2UK22dtC3Hyq8PfT385cQKDAqSOoDLgN8HlgJ3S5ofEQ/nsp0DrIyIV0g6A/g8cHpRZdoZpkzoYsqEKRx+wJSGeSKC59cPsG5D9li7YYB1Gzan5wHWbRxg/abNbNg0yMbNg2wcGGRDemwcqKZtrqVvHgwGBgdZPxAMDgabIxjYHAxGMDCYpQ15Tumbc48IGIwgSM9jq4FobdIssAxNy+fTkI11A1AuVXXzbR286h9v+wIgLc+/beXc+vjF1PWM42Zy7htfRpGKbCkcDyyOiMcBJF0DzAHyQWEOcFFavg74qiTFWOvTGkZSCh5d7S5KUxHBYGwJEtXnYGh6NV/tmaH5t+TLlqvBZ8t+ufTcejUPkJazpawM1AJX1PJm56aankuLLcm1/OS3DdknhpyP2nEaH49h24bus+V8qQZ1j1fLkq/PsPrnX5Ph58y/JtXzMCwt/7fNHyNXvJbHqC3mj7GN+0YMzZPfmC9u/rXd1nMNP0ar8w9/HYYer06+oupa5/yt8lVXpk0aR9GKDArTgSW59aXAaxvliYgBSauBfYDfFlguSyTRIegY+h3FzEpsTFySKmmupD5Jff39ntHUzKwoRQaFp4CZufUZKa1uHkmdwBSyAechImJeRPRGRG9PT09BxTUzsyKDwt3AwZIOktQNnAHMH5ZnPvCBtPwu4KdjfTzBzGwsK2xMIY0RfBy4keyS1Csj4iFJFwN9ETEf+AbwbUmLgRVkgcPMzNqk0N8pRMQCYMGwtAtzy+uBdxdZBjMzG7kxMdBsZma7hoOCmZnVOCiYmVnNmJsQT1I/8Ovt3H0a5fthnOtcDq5zOexInV8aES2v6R9zQWFHSOobySyBuxPXuRxc53LYFXV295GZmdU4KJiZWU3ZgsK8dhegDVzncnCdy6HwOpdqTMHMzJorW0vBzMyacFAwM7Oa0gQFSbMlLZK0WNL57S7PtpJ0paRlkh7Mpe0t6SZJv0rPe6V0SfpKqutCScfm9vlAyv8rSR/Ipb9G0gNpn6+ozTfslTRT0s2SHpb0kKQ/S+m7c53HS7pL0v2pzn+X0g+SdGcq57+nWYeRNC6tL07bZ+WOdUFKXyTprbn0Ufl/IKlD0r2Sbkjru3WdJT2Z3nv3SepLaaPjvR0Ru/2DbJbWx4CXAd3A/cBh7S7XNtbhd4BjgQdzaV8Azk/L5wOfT8unAP9JdovXE4A7U/rewOPpea+0vFfadlfKq7TvyW2u7/7AsWl5MvAocNhuXmcBk9JyF3BnKt+1wBkp/XLgT9PyR4HL0/IZwL+n5cPSe3wccFB673eM5v8D4JPAd4Eb0vpuXWfgSWDasLRR8d4uS0uhdr/oiNgIVO8XPWZExK1k04vnzQG+mZa/Cbw9l/6tyNwBTJW0P/BW4KaIWBERK4GbgNlp254RcUdk76hv5Y7VFhHxTET8Ii2vAR4hu33r7lzniIi1abUrPQJ4M9k9zGHrOldfi+uA30vfCOcA10TEhoh4AlhM9j8wKv8PJM0A3gZckdbFbl7nBkbFe7ssQaHe/aKnt6ksO9O+EfFMWn4W2DctN6pvs/SlddJHhdRFcAzZN+fdus6pG+U+YBnZP/mcxEhEAAAEQElEQVRjwKqIGEhZ8uUcco9zoHqP8219LdrtS8BfA4NpfR92/zoH8BNJ90iam9JGxXu70Psp2K4TESFpt7u+WNIk4HvAn0fE8/mu0d2xzhGxGTha0lTg+8ChbS5SoSSdCiyLiHskndTu8uxCJ0bEU5JeAtwk6Zf5je18b5elpTCS+0WPRc+lpiLpeVlKb1TfZukz6qS3laQusoDwnYi4PiXv1nWuiohVwM3A68i6C6pf4PLlbHSP8219LdrpDcBpkp4k69p5M/Bldu86ExFPpedlZMH/eEbLe7vdAy674kHWInqcbACqOth0eLvLtR31mMXQgeZLGTow9YW0/DaGDkzdFVsGpp4gG5TaKy3vHfUHpk5pc11F1hf6pWHpu3Ode4CpaXkC8DPgVOA/GDro+tG0/DGGDrpem5YPZ+ig6+NkA66j+v8AOIktA827bZ2BicDk3PLtwOzR8t5u+xthF/4hTiG7guUx4NPtLs92lP9q4BlgE1kf4Tlkfan/DfwK+K/cG0LAZamuDwC9ueN8iGwQbjHwwVx6L/Bg2uerpF+7t7G+J5L1uy4E7kuPU3bzOh8J3Jvq/CBwYUp/WfonX5w+LMel9PFpfXHa/rLcsT6d6rWI3JUno/n/gKFBYbetc6rb/enxULVMo+W97WkuzMyspixjCmZmNgIOCmZmVuOgYGZmNQ4KZmZW46BgZmY1DgpWOpLWpudZkv54Jx/7U8PWb9+ZxzcrmoOCldksYJuCQu5Xto0MCQoR8fptLJNZWzkoWJldArwxzWn/F2kyuksl3Z3mrf8TAEknSfqZpPnAwyntB2kys4eqE5pJugSYkI73nZRWbZUoHfvBNM/96blj3yLpOkm/lPSd6tz3ki5Rdj+JhZL+aZe/OlZKnhDPyux84C8j4lSA9OG+OiKOkzQOuE3ST1LeY4FXRzYtM8CHImKFpAnA3ZK+FxHnS/p4RBxd51zvBI4GjgKmpX1uTduOIZum4WngNuANkh4B3gEcGhGRJsgzK5xbCmZb/AHw/jR19Z1k0w4cnLbdlQsIAP9H0v3AHWSTkh1McycCV0fE5oh4Dvgf4LjcsZdGxCDZdB6zyKaEXg98Q9I7gRd2uHZmI+CgYLaFgE9ExNHpcVBEVFsK62qZsime3wK8LiKOIpuvaPwOnHdDbnkz0BnZvQKOJ7uRzKnAj3fg+GYj5qBgZbaG7FafVTcCf5qm7EbSIZIm1tlvCrAyIl6QdCjZbJRVm6r7D/Mz4PQ0btFDdnvVuxoVLN1HYkpELAD+gqzbyaxwHlOwMlsIbE7dQFeRzeM/C/hFGuztp/5tDH8MfCT1+y8i60KqmgcslPSLiDgrl/59snsj3E82++tfR8SzKajUMxn4oaTxZC2YT25fFc22jWdJNTOzGncfmZlZjYOCmZnVOCiYmVmNg4KZmdU4KJiZWY2DgpmZ1TgomJlZzf8Hga34d59GFkYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot gradient descent progress.\n",
    "plt.plot(range(num_iterations), cost_history)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.title('Gradient Descent Progress')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot the Model Predictions\n",
    "\n",
    "Since our model is trained now we may plot its predictions over the training and test datasets to see how well it fits the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xlc1NX6wPHPAREHN1AxFfcN00xRMsstbUEyFc2bme37rV+leb1hy1Wre7NL+73durZa2dVSI3c0t9I0Q0EJFTU1E/cFTRlwgPP7Y2ZwgBlmWIbZnvfr5Uv4fr/MnC/LM2eec85zlNYaIYQQ/ivI0w0QQgjhXhLohRDCz0mgF0IIPyeBXggh/JwEeiGE8HMS6IUQws85DfRKqY+VUseVUr/YOTdJKaWVUk0snyul1DtKqb1Kqe1KqV7uaLQQQgjXudKj/xQYWvqgUqoVcBNw0OZwPNDJ8u9h4L2qN1EIIURVOA30WuvvgdN2Tr0J/BWwXXE1EvhMm20CwpVSzaulpUIIISqlVmW+SCk1EsjWWm9TStmeigJ+t/n8kOXYkfIer0mTJrpt27aVaYoQQgSsLVu2nNRaRzq7rsKBXikVBjyLOW1TaUqphzGnd2jdujWpqalVeTghhAg4SqnfXLmuMrNuOgDtgG1KqQNAS2CrUqoZkA20srm2peVYGVrrmVrrWK11bGSk0xckIYQQlVThQK+1ztBaN9Vat9Vat8WcnumltT4KLATutsy+6Quc1VqXm7YRQgjhXq5Mr/wfsBGIVkodUko9UM7lS4F9wF7gA+CxammlEEKISnOao9daj3Nyvq3Nxxp4vOrNEkIIUV1kZawQQvi5Sk2vFMLbJKdlk5SSRXaOkWClKNSaqHADk+OiSYiJ8nTzhPAoCfTC5z2fnMHsTQeLV+4VWnZNy84xMmVBBoAEexHQJHUjfFpyWnaJIF+a0VRIUkpWjbZJCG8jPXrhU2xTNAocBnhbh3OM7m6WEF5NAr3wGclp2UxZkIHRVAi4FuQBWoQb3NcoIXyABHrhE5LTspn01bbi/LurDCHBTI6LdlOrhPANEuiF17P25Csa5GXWjRBmEuiF10tKySpO17gi3BBC+tQq1dwTwq/IrBvh9SoymBoSrJg2opsbWyOE75EevfB6LcINZLsQ7CPCQpg63Bzk+81YzeEcIy0kfSOEBHrhnazTKA/nGGloCHF4XVS4gQ2JQ0p8ne3MHFk0JYSkboQXsgbr7BwjGsgxmuz+otqbUWMvn280FTJtYab7GiyEl5NAL7yOvWBdBBhCgogKN6Aw9+RfGd29TC/dUT4/x2giOc3uHjhC+D1J3Qiv4yhYG01FTvPt5eXzk1KyJH0jApL06IXXKW8lq7O6NeUtjnJlQFcIfySBXngdZ8G6vBRMQkwUEWH2B28VSPpGBCQJ9MLrlBesAaYsyCg3YE8d3g1l57jG+TsCIfyRBHrhlaYO74YhJNjuOWelhxNiohwWPJNKliIQSaAXXsc6h768sgfOAnaUgzy/xryYSlI4IpBIoBdexXYOfXmclR6eHBft8B2BdRGVBHsRKJwGeqXUx0qp40qpX2yOJSmldimltiulvlFKhducm6KU2quUylJKxbmr4cI/uVLAzJXSwwkxUbwyurvDnr3sPCUCiSs9+k+BoaWOrQSu0FpfCewGpgAopboCtwPdLF/zH6WU/W6VEHY46sk3yDvPyB1reSH1K+afXkPCgc3wxx/lPlZCTBQbEofYHZgFydeLwOF0wZTW+nulVNtSx1bYfLoJGGP5eCQwR2udD+xXSu0F+gAbq6W1wq8lp2WX2R7QcDGPpzZ8yX1bFxNacBGUglUa/gPUrw9PPAHPPgt16zp8XEeLqGTnKREoqiNHfz+wzPJxFPC7zblDlmNCODV9UWaJIN8y5yjffvY0j25ewLGhI2DTJsjLA6MR1q6Fm2+Gf/wDevaE7dsdPu7gLpF2e/W5FwskTy8CQpUCvVLqOaAAmF2Jr31YKZWqlEo9ceJEVZoh/EByWjZnck3Fn7fMOcr82X+l6YXT3DH2ZVov+hquvhpq14Y6dWDQIJgzB1avhtxc6N8fvvvO7uPO35Jtd7rlmVyTDMqKgFDpQK+Uuhe4BRivdfEeb9lAK5vLWlqOlaG1nqm1jtVax0ZGRla2GcJP2A6MNsg7z+dfvUBowUX+dMer/NbzGsdfOHgw/PQTtG0Lt9xi7umXetzyBndlUFYEgkoFeqXUUOCvwAitda7NqYXA7UqpUKVUO6ATsLnqzRT+rnhgVGteXfYOLc8e5/5bp7Inso3zzb1btoQ1a6BDBxgxAtLTyz6uK88thJ9yZXrl/zAPpkYrpQ4ppR4A/g3UB1YqpdKVUu8DaK0zga+AHcBy4HGtteubfYqAZR0YvWPbcuJ3/8g/B97D1paXE24Ica3iZOPGsGIFNGwIo0bBqVMlHteV5xbCXzkN9FrrcVrr5lrrEK11S631R1rrjlrrVlrrnpZ/j9pc/3etdQetdbTWell5jy0EmPPoF/ILiDx/msQ1n7C+TQ8+7JOAISS4Yvu/RkXB/Plw+DDccQcUFpa7cApcm5MvhK+TlbHCo6wrYXOMJqas/YTQwos8f9NjhNcNtbuxiFN9+sC775p796+9VrxwKtzOdoQKuLV3lNSoF35PAr3wqOmLMjGaCul5OIvRmWv4oM9oDjSKIqx2rcoH4AcegDFj4IUXYNs2EmKiqBtadsmIBtbskhlfwv9JoBceYzulcuL62ZwyNOC9q81r76o0QKoUvPceNGoEd90F+fkOH08GYkUgkEAvPMY6rbHXoZ0M2r+VmVeP5kJoGFANA6RNmsBHH0FGBvz97w4fTwZiRSCQQC88xtqbnrh+NifDGvJZzC3F56plgHTYMBg/HmbMYHrn4DKDsjIQKwKFBHrhMS3CDXQ7upcBv6Uzs89ojLXrALg+pdIVr78OdetywztTeWXUFUSFG1CY69VXarBXCB8kgV54zOS4aB7auogLIXWY08Nc0brCUyqduewyePVVWLeOhIxVbEgcwptjewIwYW46HaYspW3iEtmMRPg1p9UrhXCXhObBFO76gW9ib+aPOvWICjcwOS66+nvZDz4Is2bBpEksad2bKasPFZdFKLRU77BuRgJIL1/4HQn0wnPef59g00XGfPoqYzp3dt/zBAXB++9DTAymxESMg/9s9zJr3RsJ9MLfSOpGeEZBAcycCfHx4M4gb9W9Ozz5JCM2L6XHYcdFzGS6pfBHEuiFZ6SkwJEjJDbuS7uaypFPm8bp+hG8uPJ9gorsl2CS6ZbCH0mgFx5x+I13ORXWkHnNeqCpoQ27GzTgQOJ0ehzdw7htKWVOy3RL4a8k0Iuad+IEketWsqDbYAqCLw0T1URt+Ngpj3Mi9hqe+eFzGuWeJViZ956S6ZbCn8lgrKh5s2cTUljA191vKHPK7TlypYic9SH06MHW3NXwzofufT4hvIAEelHjzr77Xw4078TuyLZlztVIjrxrV5g4EZKSWNd/BM8ercfhHCMt3DW9UwgPk9SNqFmZmTTcu4v53YaUOaWoptIHrnjhBYxNmxGZOJEjp8/X3DiBEB4ggV7UrLlzKVRBLIvuX+aUpgYXK9Wvzz+uf4iux/YxPv3S/jiyh6zwRxLoRc3RGubOJb3dlZyoF1HmdFQNT238olUffmjTk8nff06TC2eKj2fLXHrhZyTQi5qzbRvs3k3w7bd7RSXJFhFhTL3xUeqY8klc+2nxcQWSvhF+RQK9qDlz50JwMD0nPsAro7t7vJLk5Lho9jduyQd9RjHml1XEHsoEzCkkSd8If6K0paiTJ8XGxurU1FRPN0O4k9bQsaP5X0rZxUqe0jZxCYaLeXz34Z85V6cut9z7NoVBwShg/4xhnm6eEOVSSm3RWsc6u85pj14p9bFS6rhS6hebY42UUiuVUnss/0dYjiul1DtKqb1Kqe1KqV5Vuw3hN7ZsgX374LbbPN2SEqLCDRhr1+HF6x/i8hMHuHvrYkBKIQj/4krq5lNgaKljicAqrXUnYJXlc4B4oJPl38PAe9XTTOHz5s+H4GAYNcrTLSlhclw0hpBgUjpfw9p2vZn4w2xa552VUgjCrzgN9Frr74HTpQ6PBGZZPp4FJNgc/0ybbQLClVLNq6uxwod9+y0MHGjesNuLJMREmccLIsKYfsPD1Ck0MXv3PFk0JfxKZVfGXqa1PmL5+ChwmeXjKOB3m+sOWY4dQQSuPXtg50545BFPt8SuhJgoS2AfAs0O0eqll2DpUrj5Zk83TYhqUeUSCFprrZSq8IiuUuphzOkdWrduXeHnTU7LJikli+wcI8FKUai1+3YoElWzcKH5/5EjPdsOVzz7LCQnw333wfbt5q0IhfBxlZ1eecyakrH8f9xyPBtoZXNdS8uxMrTWM7XWsVrr2MjIyAo9eXJaNlMWZJCdY6TNmcNltoOTOdBe5ttv4coroW1bT7fEuTp14Msv4exZuP9+82whIXxcZQP9QuAey8f3AN/aHL/bMvumL3DWJsVTbZJSsjCaCknIXMOamY/Q7divxeeMpkKmLcys7qcUlXXyJGzY4Bu9easrroDXXjOnb955x9OtITktm34zVtfcBi3C77gyvfJ/wEYgWil1SCn1ADADuFEptQe4wfI5wFJgH7AX+AB4zB2NtpayXdWxD2fr1OOv62aVOJ9jNMkfg7dYvBiKimDkSN8KWI8/DiNGwF/+AuvWeaQJyWnZ9Jy+gglz08nOMUrhNVFpPrlgqt+M1cX1SB7cvIDn13zMuNv/zsY2PYqvCVaKIq2l9KynjRoFqakkf7uRKd/8gtF0aQs/Q0iwd2/2cfYsXH01nDplXgdQibGkyrKmJ22/X7aiwg1sSCxbAVQElmpbMOWNrHOfAT7vdQvZ9SN5Zt2nJfKphVpLD8jTjEZYsQJGjCBpxe4yQcubKkVa3220TVxCu8QltE1cQttX1pNw02RMefkwbBicLj3L2H2s6UlHZBNzURE+GeiL5z6HG8ivVZu3+t9BzyN7iM/aYPd6bwooAWXtWsjNheHDHVaE9IZKkbaD+2CudWOVHtaM+0dMoTBrt3m65fnzNdImZ98XWbkrKsInAz2Yg/2GxCG8NbYnS2NuJKtJayZ//xkhhSa710sPyAOWL4c6dVjYsCPKwSXeUCnSWe/5h1ZX8tzY5yA1FYYPh3PnqvX5bd9NdJiylLaJS8q9XjYxFxXls4HeKiEmir+P6ck7cQ/T/sxh7ktdaPc66QF5wLJlMHgwr647iKORIG+oFOlKJ2BOVG+mjkmkaP16uO46OHasWp679LuJQidjZhFhId49riG8ks8HeqvV7XqzsmMfnvxxDk3/OFXinML8VtjrZ3r4k19/Na+IjY93Gkg9/W7L1U7ArLbX8OiYv1GwcxfExsLGjVV+bmfvJkqbOrybBHlRYX4R6K1/LC8NeYiQwgKmrP2k+JziUs5VBmZr0DLL9nzx8U4DqaffbdkO7juzok0vHnrwDQgJMdfueekluHix0s9d0Rc5+f0VleEXgd76x3Iwojkz+4xm1I619PndXFW59BthGZitIcuWFdefH9wl0mGO3hvyzbaD+4DDtlqtrdsKtm6FW2+Fv/0NevY0r/6txFTl8LCQCl1vNBUy6attvrEWQXgNvwj0tj3C//T9EwcbXsaMZe9Qx5Rn93pvmOnh1/LyYM0aiI8nOS2b+Vuy7eboPbWzlD3Wwf0DM4axf8YwDswY5nAP2xbhBggPhzlzYMkSc48+IQFiYuCTT+CPP1x6zuS0bM7nFVS4rTJ1WFSUXwR62x6hsXYdnol/ivZnDjPphy/sXu8NMz382rp15jn08fEOc9DWBT/eEOTtSU7L5kJ+2SBc5h3IzTfDrl0waxbk55vr4zRrBn/6E3z0ERw65PA5klKyMBVVbcGivEMVrvCLQJ8QE0WEzVvgjW2u5Iue8Tzw87f0yt5Z5npvmOnh15YtMxcHu+46hzloTw/Alsc6EybHWHKqrsMZL7Vqwd13w44d8OOPcNdd5oHaBx+EVq3g8svNJRUWLCix6Kq6vgfyDlU44xeBHsyzEWwH1F657j4ON2jCa0vepF5+bpnrvTnQ+Lxly8xTEA0GhwOtnh6ALY+jdyFhtWuV/w5EKbjmGnj/ffj9d8jIMBdHa9fO3OO/9VZo0gR694bJkxl1LMPhug9HD2/3OPIOVZTPbwJ96QG1C6FhTLxlEq1zjvLK8n+VGSgLUqrEgJZPFdzyZvv2we7dMNS8+6S9GS3eMABbHkedgAr1nJUyV8GcNMlcBfPMGVi/HqZNg/r1KXz7Hd74dAo/vncff/n+MyLPOy6vYAgJ5s6+ranlINLLO1ThjE8WNXNFzIsrOJNr4s+bvuaZdbN4/qbH+CLG/o5BIUEKFJgKL30vvL7glrf6z3/MaYqsLOjcGbi0SczhHKNPFJmzLZpnT1U3uElOy2b63J/ptXcrt29LYcivqVwMDuGT2OH8+5qx5Na+9G7H+lzWTXYcUcD+GcMq1R7hu1wtaua3gd6aZ827aOLjedPpd2Abd419iZ9ad3f5MaRCYCUMHw47dpD89TqSVuz2meBuKzktm4lz0x2u5oXKdQSS07KZtjCzTO6/9ZkjTFw/m5E71vF7+GVMvnkCGe17lHj8dolLym1PuCGE9Kk3udwW4R/8unqlK6ypnBYRdZkwfDK/RTRn5oKX6XDyd+dfbCF5/ArKy4PVq9kXO4Ap3/xSoob6hLnpxLy4widSYgkxUeUGVaj4bJfktGwmf72tTJAH8/qPicP/wm3jZ6BRzPnfFL6+8CMJPVsUX+NsTCPHaPKZ76+oeX4b6OHS3Ohtb43lvj9N42JwCJ/Om0bzcydc+npvHjD0Sj/8ALm5/MfQ2e5g5plck8/M+3Y0h95WRToCrkylPHJFLG1/303Q6NFc8eZL8Oij5k1bcG31ri99f0XN8utAb0u3acsDY/5GQ+MfzPnfFFqcO17u9d4+YOiVli2D0FCWNHb8ffOVed+uBNaKdAScvSgU/77VqwdffQXPPAMzZ8Ijj0BRUZnJBo74yvdX1KyACfST46LJaN6Zu8a+RITxD+Z+OYX2p0ouZglWCoV3rdj0KcuWwaBBNIqMKPcyX0iJOSuLUNGOQHkvCsFKlfx9CwqCGTPguefgww/h6aeL27QhcYjTYO8L319RswIm0CfERDG+b2u2t4hm/NiXMZjySP58EgP2bwXMf8iFsvVg5R04YF4hOnSo096wr6TEbMsivDm2J+GGS4vy6oRU7E9ncly0eXZXKSHBitdv62H/9+2ll2DCBHj7bfjggxKP5Q/fX1FzAibQA7yc0J03x/bk9OVXknD3GxxuEMmnX0/jqfVfElxoXu5urR/yfHKGzKuviJQU8//x8cW9YdvAaOXLKbH8gqLij13Nh1vXZ0ycm07d0FqE2bxARISFkDTGQZAH81z8114zr0l4/HHYYN5BrbzvL0DuxQL5fRUl+O30SpecP0/KgATi0leR3rwTz8Q/RVZkW7uXyrx6J0aNgrQ02L+/xBJOX5tD74ijufWOpuA6mkpZqd+jM2egTx9zLZ2MDGjY0D3PI3xOjUyvVEpNVEplKqV+UUr9TylVRynVTin1k1Jqr1JqrlKqdlWew63q1eORuIk8NjKRNmeOsvSTJ/l7yr9pfCGnzKUyyFWOixdh1Spzz1OpEquMk1KymBwXzf4Zw7y6iJkzFVkt66hWDlwqM1yhHndEBHzxBRw+DE8+WeJUQkwUdUNr2X0e+X0VVpUO9EqpKOBJIFZrfQUQDNwOvAq8qbXuCJwBHqiOhrpDclo2CljapT/XPTyTz3oN47btK/n+vw/y/KoPykzDzM4xyltiezZuNJfmjY/n+eQMJs5NLzGH3h+m/DnKe9urM+Ns16hCrZk4N522FUkLXn21eXD2s8/gm29KnPLFwnGiZlU1R18LMCilagFhwBFgCDDPcn4WkFDF53CbpJSs4oUxZw31mX7DI8Td/y7Lo6/l3i2L+P6/D/L64tfpduzX4q/xh6BV7ZYtg1q1WNy4C7M3ld0f1h96l5Pjou1uSKKBCXPTizf1dlY+wfbroIIvhM8/Dz16mHv1Fy4UH/bFwnGiZlU60Guts4HXgIOYA/xZYAuQo7W2FvI+BNh9r66UelgplaqUSj1xwrUFTNXNXo9nX+OWTBr2NIMe+ZAvYm5m6O6NLPn0KeZ8mchNuzeSn3/R54NWtVu+HPr355X19jcYAd/vXTpbLWvd1Ds7x+h0h6rSXH4hDAmBd98117h/+eXiw/Z28PLlQW9R/aqSuokARgLtgBZAXWCoq1+vtZ6ptY7VWsdGRkZWthlVUl6PJ7thU6bf8AjXPPYpLw++n5ZnjzHzm7+z+oNHufG7uS7vIuT3Dh+Gbdtg6NByg7k/9C5dWS0LZbevdIXLL4T9+sG998Lrr0NWlt0dvBRwa+8onx0PEdWvKqmbG4D9WusTWmsTsADoB4RbUjkALQGvzXO4svrxXJ16fNhnNIMe+ZA/j0zkZN1wpq2aybkmzfj82ltZsWRTDbXWS61YYf5/6FCH+58q8IvepaP0TXWo0Avhq69CaCg8/7zd8QANrNnlmXfJwjtVJdAfBPoqpcKUUgq4HtgBrAHGWK65B/i2ak10H9vVj9YVsW+N7Wl3v9DCoGCWdenPmDuTGHnX66zpcBW3//Qt1w/vx+83j4Zsr309c6/ly6FZM5ILGzvc/3R839Z+0bt0pdhZeVSp/60qnGZp2tS8WnbePBrv3G73El9PlYnqVaV59Eqp6cBYoABIAx7EnJOfAzSyHLtTa51f3uN4bB59OaxT5MqbPdHs3Enu3bqI+7YsJLROKLz4Ijz1lHkJeyAoLITISBg5kn7Rd9kdhPS38rmuDrba89bYniTERFXP2oJz56B9ezZHtOW2W6eXOS0ltgNDwNejrw7JadlM+mpb8UCbI61zjvL9oW9gyRK4/nrzFLgWLcr9Gr+waZN567w5c2iXVs9ub9ffNsRwpQNgj1sC7xtvwKRJ3HPnDNZFXVHiVERYCFOHd/OLd1LCsYCvR18dEmKieP22Hs7z+M1bwaJF5nokGzeap8BZlqv7teXLze9ebrghYKb4uVpF0pbbZsA89hg0b07SniVlyiFIyWJhSwK9E87qioBl04eXVpLcOx62bDGvZLz+epgzpwZb6gHLl5uX5jdu7JN7w1aWq1Ukwc2VUOvUgYkTafrTD/Q+8WuZ0/6wfkFUDwn0LkiIiSJ96k28NbYnDvZnvtSDMtY39+qvvhrGjYP//rdmG1tTTp6EzZuLNwG3N7Dt77VWHM3aiggLKR7Ud3vZh0cegfBwxqz8wu7pyo4nCP9StkiGcCghJoqJc9MdnjeaCpkwN53pYSFM/+cnjHjpSfMuQUFB8NBDNdjSGrByJWgN8fHFhxJiAmvutvVePVq0rUEDePxx4v7xD9qfOsS+xi1LnLaWaAikn4soSwJ9BbUINzjtJZ3JNTFp4S544R1GaA0PP2zeOWjcuBpqZQ1YtgwaN4bevT3dEo/yihe3J59Ev/YaD/6czLND/6/EKY35hcjjbRQeJambCnI152wq1Ly65gDMnw8DB5pXM65f79a21ZjCQli6FOLjSd5+VOr2e1rTpgTfeScJO9bQIO98mdMyp15IoK+ghJgoIhysAC3tcI7RPGD2zTfQti0kJMDeve5tYE3YtAlOnWLzFf2YsiCjRKXKiXPTeT45w9MtDDyPP06YKZ8/ZXxX5pS/zXwSFSeBvhKmDu/mdMolQJBS5p7uzHTemPAGZ3MvsqPfTQx+aZlv93wXLYJatXje2MLu8vvZmw769v35opgYTvW8irvTlqB0UYlTsuOUkEBfCa7OpS7Uurin+85v8NSwSXQ9vp+H573l23OcFy2CgQPZk2f/xc6aFxY1q/EzT9PmzBFuzi5ZFkHm1AsJ9JVku3H0W6U2jnZkbYdY/n3NbYzbvoK49O98Mxju2wc7dsAtt5SbEpC8sAeMHg3NmnFH6qIyp2ROfWCTQF8NbOfZO0vpvNl/PJtbduXFFe+hDx6soRZWo8WLAVjZ/iou5NsvYgaSF/aI2rXh4Ye5JmszUWePlzktL76BSwJ9NZq+KNNpDZTCoGAmDXuaYF3E2yv/BUVF5V7vdRYv5o+2HXgy9bzdPVHBf1fE+oT77wcFt/6yqswpefENXBLoq0lyWjZncu0HvtJ+D2/GP294iKv2boX333dzy6rRuXOwdi2LWvV2+IIWCCtivVqbNpzs05/bMr4rMSirMO9EJQKTLJiqJhXJf0aEhRDz8l+hMAsmT2ZFm15Mz8j13OpKFySnZfPz6x/wd5OJb1vG2L1GgZTG9QJNn/oz3HEH1/62nQ1tewLmAfL5W7KJbdPI6363hPtJj76aVCT/GVa7Fgm9WsIHH1CgIWTiU2SfyS2eoeNtMySspXl7bfuBs6F1SW3Z1e51DV0YkBY1YNQoztWpx58yVpY4LAOygUsCfTWpSP6zuIRC69a8N/guBu/ZTNzujcXnve0PMikli4K8PG7Y8xPfdbqawiD7A84XZL62d6hTh+Sug4jP+rHMSlkZkA1MEuirib1KhuXtL9o2cQltE5fwVreb2dG0HdO++y9183OLz3vTH+ThHCPX/LadhvkXWNa5n8PrTIXaq16gAtmaa28htNDEiJ3flzgu77oCkwT6amKvTO/4vq2dbiZdGBTMs3H/x2XnT/P0+tnFx1uEG0hOy/aKOjItwg3EZ23gfG0DP7Szn5+38qYXqEA28r5b2Nm0HWNKpW/kXVdgkkBfjayLqPZb6pC/nNDdpc2k01tE87+ecdyzZREdTx7EEBLM4C6RZerIeCp3/9frOxC3dxOrO1xFfq3a5V4rU/i8Q0KvlizreT09j+yh7elLvzPyriswSaB3M1e3nHt9wF1cqG1g6qoPqFNL8cWmg2WmMHoqdz/y/D4a5Z7jp16Dy71O5s97l6869qcIxcgd60ocl3ddgUcCvZs52oWotNNhDXmr/x0MOJBGrwzH+8165I903jwwGOj72HiCHWyxFayUzJ/3MsGtWrGpdXdG7lhr3iTGQt51BZ4qBXqlVLhSap5SapdSaqdS6hqlVCOAjTiTAAAbLUlEQVSl1Eql1B7L/xHV1VhfVJHNpD+PGcaexq14fvWH1C6wv/iqpv5IreMD7Z9ZxPHP5pDSphdPLNxNoS6bjDKEBPP6bT0kyHuZyXHRLO0+hPZnDnPl0T2AvOsKVFXt0b8NLNdadwF6ADuBRGCV1roTsMryeUArXQDNEGL/214QXIuXhjxIuzNHuHfLwjLna+qP1DpvPjvHyNUHf6Hp+dMscjDbRnry3ishJopr/voQF4NDSMhcK6uWA1ilV8YqpRoCA4F7AbTWF4GLSqmRwHWWy2YBa4FnqtJIf5IQE0VSSpbD7Qi/b9+blR378MSPc/im2xBO1DO/IYqyrJgF6Dl9RXGdmYiwEKYO71atf7xJKVnF4wOjMlfzR20DKztebffaIq0lcHixYQO7wohbuP/HH7l/8iAIdp5GFP6nKj36dsAJ4BOlVJpS6kOlVF3gMq31Ecs1R4HL7H2xUuphpVSqUir1xIkTVWiG73GWZ//74AcILTAx+ftZGEKCeWtsTybHRTNtYSYT5qaXKCZ2JtfEhLnpxLy4otpm5FjbF2rKJz5rA8ui+5EfEmr3Wsn3er/N1w6FY8e4645XZLvHAFWVQF8L6AW8p7WOAS5QKk2jtdZgf4ah1nqm1jpWax0bGRlYxZacBccDjaL4OHYEt2V8x3vR5p71lAUZDqtFgjngV9c2ftb23bj3J+pfNPJNN/uzbSTf6/2S07J56FQzzoXWZeSOtWTnGKu9YyC8X1UC/SHgkNb6J8vn8zAH/mNKqeYAlv/LFsYOcK7MxPn3tbdzom44V/7zb0ycm+a0/DFU3zZ+1vYlZK7hcP0mbGrdvcw1EWEhku/1AUkpWZzVwSzrfC1xu38k1JQPyK5TgabSgV5rfRT4XSll7dJdD+wAFgL3WI7dA3xbpRb6odKraO1NWTwfGsarg+6lUcZWRmaudfmxy9vGz9lKW+v5iXPTaZZ/jkH7t7Kw6yC0uvRrooA7+7Ym7W83SZD3AdY03MKug6h/0ch1+7cUn/O2mkrCfapapvgJYLZSqjawD7gP84vHV0qpB4DfgNuq+Bx+KSEmqjhQJqdlM3Fuepkc1/wrhnBn2hIS137Cik59ya3tWj48O8dIvxmrS5Q9BnP6x/rOwLrS1toW60wb6/kbf15OSFEh+ePvJOqCwatLKAvHWoQbyM4xsql1d04ZGjBs13pSOl9bfF4WTwWGKk2v1FqnW/LsV2qtE7TWZ7TWp7TW12utO2mtb9Ban66uxvqrhJgouwMZWgUx/fpHaHb+NI9v/Mrlx1NQpnSCvd2vbHt0tueVLmLctuX81LIbn5wOK1HWQYK8b7Gm4QqDglkefS3X791MHVNe8XkZTA8MsjLWSzhaUJUW1YX53Qbz4M/f0PrMEbvX2FKUHf02mgod7n51OMdYZnesa3/bTrszR5gdE0+O0VQtA7zCM6xpwnBDCIu7DKCuKY/r9pnTNzKYHjgk0HuJ8gZoXx10LwVBtXh+zUdOH8eVImq2gpRiwtz0EsfGpy3llKEByy2LpKpjgFd4jnXz+tufHs/peuHcsmu9LJ4KMBLovYS152VvYPZ4/ca8e81t3LRnE0OPbCcizH5N8XBDiNOyyKWVLmnQ4txxbtqzia+738DFWubnKW+AV/iOkVe1ISd+BNf/+jNnjp8hKSVLXsADhAR6L5IQE0WRnVoyAB9dlQDt2/P+z58zbWjnMr1/Q0gwSlW8R1/ag5uT0Uoxq/fwEsdl0M73JadlMy20GwZTHtftS/XKbSuFe0ig9zKOBseaNGkIb74JO3cycvWcMpuc3No7ymEe3lXhxnPcvj2FhV0HcaRByUVsMmjn+5JSsljfvAvH60YwbNcPgEyxDBQS6L2MvVx98aDZiBEwejRMm0ZC3QvFs2Emx0Uzf0vVe2X3bFlMmCmf9/vcav/5hU87nGOkKCiYpdH9GPJrKmEXze/SHNVdEv5DAr2XsbclYYlBs3//GwwGePBBKCoCShYhq6wGeee5b8tCdsUO5PH/G+n4+YXPsr4rW9KlP4aCfIb8+jNgnqkl6Rv/VtUFU8INbBdTldG8Obz+OjzwALz/Pjz2WLXkz/+8aR4N8i5wX8/buAfYkDikyo8pvMvkuGgmzk0ntWVXjtVrxLBd61l8+cDiwXZ5Mfdf0qP3RffdBzfdBH/5C2RmVih/bm9WTrNzJ7lvy0KSu11HWuO2krP1U9aFeVoFsTS6H4P3pVI3PxeQwXZ/J4HeFykFs2ZB/fowdiyJA1uXe7ltGmZ839ZlxgCmrpoJwBsD7gTkj96fWRfmLe4ygDoFF7nekr6RwXb/JoHeVzVrBl98AZmZDP/wH0QY7GfhosINJUoYvJzQvcTWhnFZPxK/+0fe6ncHhxqatw6QP3r/ZR3s3xrVhSP1GjNs1w8y2B4AJND7shtvhBdegE8+4YvT6xzP1inFurXhe0Oa8/LK98hs2p4Pr0oo92uEf7AO9jcMC2Vpl/5ct28Lwef/YPqiTBmQ9WMS6H3dtGkwbhzd3nmFL2pnuT5bJj+f+JcnEFGYxyt3PEdhcC2ZYRNALuQXsKRLf0ILTdyw9yfO5JqYPG+bBHs/JbNufF1QEHzyCRw9Su+pE9nw3nuQ+Ej5X1NQAHfcAT/8QK0vv+SLceNqpq3CKySlZGEq0qS1iCa7fiS37Pye5G6DMRVqmX3jp6RH7w9CQ2HJErj5Znj0UXjkEcjNtX/tqVMQHw8LFsBbb4EE+YBjHWzXKojFlw9g4P40GuSdL3FO+BcJ9P7CYIBvvoFnnoGZMyE62lwyYe9ec9Dfuxf++U/z8XXr4MMP4amnPN1q4QG2g+2LuwygdlEBcbt/LHNO+A8J9P4kJARmzIDvv4c2beDpp6FTJ6hb1/z/M89Ajx6QmmpecCUC0uS4aEKCzCsqMpp15EB4c4bvNNe+yb1YIHl6PyQ5en80YACsXw+7d5uD/okT0LQpDBxoDvgioFlz8NMWZpJjNLH48gE8umkejXLPcpqGJbaYFP5BevT+rHNnc02cKVPMPXgJ8sLCuhlJVLiBRZcPpJYuIj5rAyAVLf2RBHohAtjhHCNZTdqwp3Erhu/8vsRx4T+qHOiVUsFKqTSl1GLL5+2UUj8ppfYqpeYqpWpXvZlCCHdoEW4ApVh0+UD6/J5J0z9OXTou/EZ19OifAnbafP4q8KbWuiNwBpBRPyG8lLUkwuIuAwhCMyxrvayO9kNVCvRKqZbAMOBDy+cKGALMs1wyC0ioynMIIdzHWhIhv0MnMpu2Z/iuH4pz9DL7xn9UddbNW8BfgfqWzxsDOVrrAsvnhwAZuhfCi1ln1yxfMYhJaz4h6uxxsmkqs2/8SKV79EqpW4DjWustlfz6h5VSqUqp1BMnTlS2GUKIapCUksU3nfsByH6yfqgqqZt+wAil1AFgDuaUzdtAuFLK+k6hJWD3/Z/WeqbWOlZrHRsZGWnvEiFEDTmcY+RQeDPSm3eW2Td+qNKBXms9RWvdUmvdFrgdWK21Hg+sAcZYLrsH+LbKrRRCuJV1ls2iywfS/divtD2dXeK48G3umEf/DPC0Umov5pz9R254DiFENbo0+6Y/RShG7lgns2/8SLWUQNBarwXWWj7eB/SpjscVQtQM64BrUkptNrbpzp92rqXdO6/KQKyfkFo3QgjAHOwTYqKg+dNw7720zDuIeZhN+DoJ9EIIktOySUrJ4nCOkQ6GJiyvY6DWrFlw7bWebpqoBlLrRogAl5yWzZQFGWTnGNHAXiMs6XQtpi/nQF6ep5snqoEEeiECXFJKFkZTYYljX3UdTMj5c2x+Z5aHWiWqk6RuhAhw9ubKb2zdnSP1GvPHzI9oe7olUeEGJsdFy+Csj5IevRABzt5c+aKgYJK7DWbQvi00uXCG7BwjUxZkSP0bHyWBXogAZ51DX9qCboOppYsYmbkWkJIIvkwCvRABzlrBMlipEsf3RLZha4to7tiWAloDUhLBV0mgF0KQEBPF67f1KNOz/7JnPB1OH+Lq338BpCSCr5JAL4QALvXsbS3u0p9zoXW5I305gJRE8FES6IUQxRJiooiy6bXnhdRh/hVDiM/aQLuiCzLrxkdJoBdClFB6cHZ2z3hqFxXwr4vbPdgqURUS6IUQJVhTOFHhBhRg7BjNyZ59uGLJXCgq8nTzRCXIgikhRBnFBc6sWj0Jd94JK1dCXJznGiYqRXr0QgjnxoyB5s3hjTc83RJRCRLohRDOhYbCE0/AihWQkeHp1ogKktSNEMKh5LRspi3MJMdooqGxHZtCQjn+3Mu0WTjX000TFSA9eiGEXclp2Uz+ehs5RhMAZw31mdv9RpovWcDyFVs83DpRERLohRB2JaVkYSrSJY59HDuSWkWFnPjH6x5qlagMCfRCCLvs1bU5GNGcpV36M2pjMpw86YFWicqQQC+EsMtRXZu3+o0jzJQHr71Wwy0SlVXpQK+UaqWUWqOU2qGUylRKPWU53kgptVIptcfyf0T1NVcIUVMmx0UTEqTKHN/bpDUZA+LhX/+C48c90DJRUVXp0RcAk7TWXYG+wONKqa5AIrBKa90JWGX5XAjhYxJiokj6Uw8MIWXDRGK3Uei8PHj1VQ+0TFRUpQO91vqI1nqr5eM/gJ1AFDASsG40OQtIqGojhRCekRATRaO6oWWO72zQnHldh3Dx7X8x6JEP6Tdjtew+5cWqJUevlGoLxAA/AZdprY9YTh0FLquO5xBCeIajzUaSBt6FKSiYZ9d8RHaOkQlz04l5cYUEfC9U5UCvlKoHzAcmaK3P2Z7TWmtAO/i6h5VSqUqp1BMnTlS1GUIIN3E0KHu8fmPeveY24vZs4toD6QCcyTXJ3rJeqEqBXikVgjnIz9ZaL7AcPqaUam453xywO1qjtZ6ptY7VWsdGRkZWpRlCCDdytKcswEdXJfBbeDNeWvkeoaZ8wLy37LSFmTXZROFEVWbdKOAjYKfW2rbS0ULgHsvH9wDfVr55QghPc7SnLEB+rdokDn2CDqezeXr97OLjOUYTzydfqomTnJZNvxmraZe4RPL5HlCVWjf9gLuADKVUuuXYs8AM4Cul1APAb8BtVWuiEMLTrCWLJ8xNL3NuY5sezO45lAd/Tial0zVsbXk5ALM3HQRg8bYjxWUUALJzjExZkFHicYV7Ka3tptBrVGxsrE5NTfV0M4QQTsS8uIIzuaYyx+vl57L0kyeoVVTILfe+zemwhk4fKyrcwIbEIe5oZsBQSm3RWsc6u05WxgohXDZ1eDfKJnDgfGgYf06YQuPcs7y9MImgokKnj+VoNo8tSflUDwn0QgiXJcRE2Z9GB2Q268jfbnyUAb+l89LK98BJtsDRbB6r5LRspizIIDvHiOZSykeCfcVJPXohRIVEhRvIdtAbn9sjjjY5R3hs0zxy6tQnaeDdYGcQNyRIkXuxgHaJS2gRbmByXHSZfH1SShZGU8l3BkZTIUkpWRXK7SenZZOUksXhHKPD5/J3EuiFEBUyOS6aKQsySgRhQ0gwvVo3ZMOvp/nnwHsIN57n8U1fYzDl8/KQBygKujQ9s3awQkNxrj87x8jkr7cxfVEmObmm4mDsKLVTXsqndFAf3CWS+Vuyi9vqykCwP74wyGCsEKLC7AXDpJSs4p6+0kU8v/ojHkj9ltXtY/nLsInFA7QKB6sobRhCgqkTEmR34Lf0Y0SEhTB1eDeAMi9Ajp7L0UCwNV1U+kXsldHdvTLYuzoYK4FeCFEt2iUuKRNU79y6hBdWf8DZOvV55br7SO52HVq5PjQYEqwwFVZ/jFLA/hnDyhzvN2O13bSUt84QcjXQS+pGCFEtWtjJ3X/RaxhbWl7OjGX/4s0lb/DoT/P4PGYYS7r054wLUzALizRhIUHkmooACCk0EZF7jkbGc0QY/6AgKIgLtcM4bWjAsfqNXH4RaWgIod+M1WXSM47SQo7GJEqrcNonPR26doXatV16/MqSHr0QolrYS3tYKV3EiB3rePDnZLof+5UiFJmXtWdn03YcDG/GhdoG8mvVxmDKp+5FI/Xyc2lkPEej3LOWoH6OiNxzNLiY6/D5jbVCORDRnPTmndnc6gp+bHMlx+o3cant1vSMbfqpRPuBN8f2LBG0nY0H2D6u3WC/eTMMGQL33Weu7V8JkroRQtQ42+DX0BDCxYLC4t44AFrT/eheBu9Lpe/BDDqcPsRl50+XeZzckFBOGxpwxvLvVFhDzhgacDrM/PlpQwNyDA0ILiqkrslIkws5tD1zmI6nfqdX9i4a5l8A4Oeoriy+fADJXa/jrKF+uW0PN4QwbUQ3Js5Nd5rXt/ei5mw8wPZ70y/vKB/NmkxoRDhs2ADNmzv71tolgV4I4RWsAc5R+qN2gYk6BfnUMeWTFxLKhdoGCoPsF1FzhdJFdDlxgOv3bmbYrvVcfuIAxlqhJHcdxKzew9nVtJ3Dr40IC3E4AGwVrBSFFYyb/To04sdfT6OBzicO8OWc5ygMrsX2z5O5cWT/Cj2WLQn0Qgiv4mig090uP76Pu7YuYVTmWgwF+azseDVv9xvHL8061nhbioN8UDC3j3uFi+07VmmQV0ogCCG8yuS4aLvlE9xtZ9P2PDv0Ca5+fBav9x/PVYcyWTxrAh/Om073I3tqrB1X/f4LX89+pjjI72/kePC3ukmgF0LUiISYKMb3bV1twb6ij3OuTj3+1W8c/R/9mKQBd9E7exeLPpvIB/NfpNvRvdXUKvtG7FjHF3Of52TdCG4d/0/2NzIPzjorA1FdJNALIWrMywndeXNsT7u17SvCEBLM+L6tiQo3oDAPeN7ZtzXhhhC714cEK+7s2xowF2B799qxDHj0I5IG3MVVh3awZNYEZi54ma7H9lWpXaWFFlzkpRX/4Z1FSaQ3j2b0nUkcCm8GmF+oJsdFV+vzOSI5eiFEjXO0ArW81bBWUeXMT3c0DhBuCCF96k12z9fPv8C9qQt58OdkGuZfYHnna3ir3x3lDtq6otehnfwj5d90Ofkb7/cZzWsD76Yg2Lx0SQHXdmjEgVPGKpVakAVTQgivZQ1opRcXQdkyBrYUlDt46Sjnfday8Ym9Oj1/hNblX/3GMav3cO5P/Zb7f/6Wobs3srp9LHN6xLG6w1XFAdoVHU79zuMbv2J05hoO12/CvWOmsrbDVcXnoypZg6cqpEcvhPAqyWnZTPpqm90pjM5KEZTXo68bWqt4fr9SkJNrsjvvvUHeee5P/Zbbt6XQ7PxpTtQNZ2XHvqxt35utUV04GRZesiKn1rQ8d5xrftvG8J0/0P9AOvm1ajOr9y28c+3t5Na+lIe3tr+6Si3I9EohhM+qbHExe18XEqRAUaJmjrOVsFHhBoILC+mU9gNjMlYx4EAa9S6arzttaMCxeo3IqxVKmMlI0/NniMj7A4DfwpuR3HUws3rfUmaXLUNIMLf2jmLNrhMOp5k6qsHjiKRuhBA+y1Fqx1law97X5V4sKJP3t9a1d1RyuTiNlFfIqo5XE1Joolf2Lroe30enkwdpnHsWgymfY/UbUXvQQP6TF86GyM7saNrObv1963PO3nSw3Mqd7pqFI4FeCOGVEmKiKpWvLv117RKX2L3ucI7RpRcUc68ffmrdnZ9ady/xGNZUy1IXF4OVF+RtX2CqmwR6IYRfs1dV03ocyn9BsZ5zlEqyBuaqLnyy1tR3V817t82jV0oNVUplKaX2KqUS3fU8QghRnslx0RhCStbOqWjvOSEmildGdy8xb992vKCqKZc828JvbuCWwVilVDCwG7gROAT8DIzTWu+wd70Mxgoh3Mnd2wNWZV2AVWU2N/H0YGwfYK/Wep+lMXOAkYDdQC+EEO5U2Xx/RR4fKr4uwJY76964K9BHAb/bfH4IuNpNzyWEEB5X3ouJs1lA4N66Nx4bjFVKPQw8DNC6dWtPNUMIIdyq9AuAs4Fdd3DXYGw20Mrm85aWY8W01jO11rFa69jIyEg3NUMIIbyLs4Fdd3BXj/5noJNSqh3mAH87cIebnksIIXyKu8cMSnNLoNdaFyil/g9IAYKBj7XWme54LiGEEOVzW45ea70UWOquxxdCCOEa2XhECCH8nAR6IYTwcxLohRDCz3lFPXql1Angt0p8aRPgZDU3x9vJPQeOQLxvueeKaaO1djo/3SsCfWUppVJdqfPgT+SeA0cg3rfcs3tI6kYIIfycBHohhPBzvh7oZ3q6AR4g9xw4AvG+5Z7dwKdz9EIIIZzz9R69EEIIJ3w20AfKVoVKqQNKqQylVLpSKtVyrJFSaqVSao/l/whPt7MqlFIfK6WOK6V+sTlm9x6V2TuWn/t2pVQvz7W88hzc8zSlVLblZ52ulLrZ5twUyz1nKaXiPNPqqlFKtVJKrVFK7VBKZSqlnrIc99ufdTn3XLM/a621z/3DXCjtV6A9UBvYBnT1dLvcdK8HgCaljv0TSLR8nAi86ul2VvEeBwK9gF+c3SNwM7AMUEBf4CdPt78a73ka8Bc713a1/I6HAu0sv/vBnr6HStxzc6CX5eP6mLcb7erPP+ty7rlGf9a+2qMv3qpQa30RsG5VGChGArMsH88CEjzYlirTWn8PnC512NE9jgQ+02abgHClVPOaaWn1cXDPjowE5mit87XW+4G9mP8GfIrW+ojWeqvl4z+AnZh3o/Pbn3U59+yIW37Wvhro7W1VWHPFnWuWBlYopbZYduUCuExrfcTy8VHgMs80za0c3aO//+z/z5Km+NgmJed396yUagvEAD8RID/rUvcMNfiz9tVAH0j6a617AfHA40qpgbYntfn9nl9PnQqEe7R4D+gA9ASOAK97tjnuoZSqB8wHJmitz9me89eftZ17rtGfta8GeqdbFfoLrXW25f/jwDeY38Yds76Ftfx/3HMtdBtH9+i3P3ut9TGtdaHWugj4gEtv2f3mnpVSIZgD3myt9QLLYb/+Wdu755r+WftqoC/eqlApVRvzVoULPdymaqeUqquUqm/9GLgJ+AXzvd5juewe4FvPtNCtHN3jQuBuy4yMvsBZm7f9Pq1U/nkU5p81mO/5dqVUqGV7zk7A5ppuX1UppRTwEbBTa/2GzSm//Vk7uuca/1l7elS6CqPZN2Mewf4VeM7T7XHTPbbHPAK/Dci03ifQGFgF7AG+Axp5uq1VvM//YX77asKck3zA0T1inoHxruXnngHEerr91XjPn1vuabvlD765zfXPWe45C4j3dPsrec/9MadltgPpln83+/PPupx7rtGftayMFUIIP+erqRshhBAukkAvhBB+TgK9EEL4OQn0Qgjh5yTQCyGEn5NAL4QQfk4CvRBC+DkJ9EII4ef+H0iUhi9QYydTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get model predictions for the trainint set.\n",
    "predictions_num = 1000\n",
    "x_predictions = np.linspace(x.min(), x.max(), predictions_num).reshape(predictions_num, 1);\n",
    "y_predictions = linear_regression.predict(x_predictions)\n",
    "\n",
    "# Plot training data with predictions.\n",
    "plt.scatter(x, y, label='Training Dataset')\n",
    "plt.plot(x_predictions, y_predictions, 'r', label='Prediction')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may see from the plot how well our model predicts the training set examples."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
